Information processing device, information processing method, and program

ABSTRACT

Provided is an information processing device including an acquisition unit that acquires a first captured image, a second captured image, and a distance to a subject, and a derivation unit that derives an imaging position distance which is a distance between the first imaging position and the second imaging position, on the basis of a plurality of pixel coordinates for specifying a plurality of pixels of more than three pixels which are present in the same planar region as an emission position irradiated with the directional light beam on the real space and correspond to the position on the real space in each of the first captured image and the second captured image which are acquired by the acquisition unit, emission position coordinates which are derived on the basis of the distance acquired by the acquisition unit, a focal length of an imaging lens, and dimensions of imaging pixels.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 15/954,627, filedApr. 17, 2018, which is a continuation application of InternationalApplication No. PCT/JP2016/081597, filed Oct. 25, 2016. Further, thisapplication claims priority from Japanese Patent Application No.2015-218642, filed Nov. 6, 2015. The entire disclosures of all of theabove applications are incorporated herein by reference.

BACKGROUND 1. Technical Field

The technique of the present disclosure relates to an informationprocessing device, an information processing method, and a program.

2. Related Art

There has been known a distance measurement device that performsdistance measurement on the basis of a reciprocating time of a laserbeam emitted by an emitting unit toward a subject assumed to be adistance measurement target by a user (see, for example,JP2012-167944A). Meanwhile, in this specification, the distancemeasurement refers to measurement of a distance from the distancemeasurement device to a subject to be measured.

In addition, as a type of distance measurement device, there is alsoknown a distance measurement device equipped with a three-dimensionalcoordinate calculation function which is a function of calculatingthree-dimensional coordinates of a subject.

As a method for realizing the three-dimensional coordinate calculationfunction, JP3095463B discloses a three-dimensional measurement method ofcalculating three-dimensional coordinates of a plurality of measurementpoints of an object to be measured from image data, which is obtained byimaging using two or more cameras, by a triangulation method.

The three-dimensional measurement method disclosed in JP3095463Bincludes the following steps 1 to 5. In step 1, four or more referencepoint members and reference bars are disposed within fields of view oftwo or more cameras. In step 2, the reference point members and thereference bars are imaged by the two or more cameras, and angles in thehorizontal direction and the vertical direction of the reference pointmembers and the reference bars with respect to an optical axis of eachcamera are obtained from image data obtained by the imaging. In step 3,a relative positional relationship between the reference point members,the reference bars, and the two or more cameras is obtained from data ofthe angles of the reference point members and the reference bars.

In step 4, absolute positions and poses of the two or more cameras arecalculated from the obtained relative positional relationship and adistance between two reference bars. In step 5, three-dimensionalcoordinates of a plurality of measurement points of an object to bemeasured are calculated from image data, which is obtained by the two ormore cameras, by a triangulation method on the basis of the calculatedabsolute positions and poses of the two or more cameras.

In addition, as a device for realizing a three-dimensional coordinatecalculation function, JP2013-122434A discloses a three-dimensionalposition measurement device including a monocular imaging device towhich irradiation means having an irradiation light source emitting alaser beam is fixed.

The three-dimensional position measurement device disclosed inJP2013-122434A captures an image of a calibration plate as a subject bymoving the calibration plate while irradiating the calibration platewith a laser beam or captures an image of the calibration plate as asubject from two imaging positions by moving an imaging device. Inaddition, the three-dimensional position measurement device disclosed inJP2013-122434A calculates three-dimensional coordinates of an emissionposition of a laser beam in each image from the captured images, andcalculates a direction vector or a plane equation of the laser beam. Thethree-dimensional position measurement device disclosed inJP2013-122434A calculates three-dimensional coordinates of an object tobe irradiated with a laser beam by using the calculated direction vectoror plane equation.

SUMMARY

However, in the technique disclosed in JP3095463B, it is necessary touse the reference point members and the reference bars, it is notpossible to calculate three-dimensional coordinates of a plurality ofmeasurement points of an object to be measured under a situation whereit is not possible to use the reference point members and the referencebars.

In the technique disclosed in JP2013-122434A, the calibration plate hasa plurality of characteristic locations, and the characteristiclocations of the calibration plate are irradiated with a laser beam.However, in a case where any subject, such as an existing building, isirradiated with a laser beam, it is considered that the subject does nothave a characteristic location to be irradiated with a laser beam. In acase where the subject does not have a characteristic location, it isdifficult to irradiate the same location with a laser beam fromdifferent positions. As a result, in the technique disclosed inJP2013-122434A, it is also difficult to calculate three-dimensionalcoordinates of an object to be irradiated with a laser beam.

As another method of calculating three-dimensional coordinates, a methodof calculating three-dimensional coordinates on the basis of a firstcaptured image, a second captured image, and an imaging positiondistance is considered. Here, the first captured image refers to animage obtained by imaging a subject from a first imaging position, andthe second captured image refers to an image obtained by imaging thesubject from a second imaging position different from the first imagingposition. In addition, the imaging position distance refers to adistance between the first imaging position and the second imagingposition.

In a case where three-dimensional coordinates are calculated on thebasis of the first captured image, the second captured image, and theimaging position distance, it is necessary to obtain the imagingposition distance with a high level of accuracy. The imaging positiondistance can be calculated, for example, when distance measurement isperformed by using a predetermined location which is a characteristiclocation capable of being specified and a subject including thepredetermined location can be imaged from each of the first imagingposition and the second imaging position. The calibration plate makes itpossible to provide the predetermined location to a user, but theimaging position distance cannot be calculated in a case where thecharacteristic location capable of being specified is not irradiatedwith a laser beam under a situation where the calibration plate cannotbe used.

One embodiment of the invention is contrived in view of such situations,and provides an information processing device, an information processingmethod, and a program which are capable of deriving an imaging positiondistance on the basis of captured images obtained by imaging a subjectfrom each of different imaging positions even when a characteristiclocation capable of being specified is not irradiated with a laser beam.

An information processing device of a first aspect of the inventionincludes an acquisition unit that acquires a first captured imageobtained by imaging a subject from a first imaging position, a secondcaptured image obtained by imaging the subject from a second imagingposition different from the first imaging position, and a distance fromone of a position corresponding to the first imaging position and aposition corresponding to the second imaging position to the subject,the distance being measured by emitting directional light, which hasdirectivity, to the subject and receiving a reflected light of thedirectional light, and a derivation unit that derives an imagingposition distance which is a distance between the first imaging positionand the second imaging position, on the basis of a plurality of pixelcoordinates being a plurality of coordinates for satisfying theplurality of pixels of more than three pixels which are present in thesame planar region as an emission position irradiated with thedirectional light on the real space and correspond to the position onthe real space in each of the first captured image and the secondcaptured image which are acquired by the acquisition unit, emissionposition coordinates which specifies the emission position on the realspace and are derived on the basis of the distance acquired by theacquisition unit, a focal length of an imaging lens used for the imagingof the subject, and dimensions of imaging pixels included in an imagingpixel group for imaging the subject.

Therefore, according to the information processing device of the firstaspect of the invention, it is possible to derive the imaging positiondistance on the basis of captured images obtained by imaging the subjectfrom each of different imaging positions even when a characteristiclocation capable of being specified is not irradiated with a laser beam.

In the information processing device of a second aspect of the inventionaccording to the information processing device of the first aspect ofthe invention, the derivation unit derives designated pixel real spacecoordinates, which are coordinates of designated pixels on the realspace which are designated as pixels corresponding to the position onthe real space in each of the first captured image and the secondcaptured image which are acquired by the acquisition unit, on the basisof the derived imaging position distance.

Therefore, according to the information processing device of the secondaspect of the invention, it is possible to derive designated pixel realspace coordinates even when a characteristic location capable of beingspecified is not irradiated with a laser beam.

In the information processing device of a third aspect of the inventionaccording to the information processing device of the second aspect ofthe invention, the designated pixel real space coordinates are specifiedon the basis of the imaging position distance, the focal length, and thedimensions.

Therefore, according to the information processing device of the thirdaspect of the invention, it is possible to derive the designated pixelreal space coordinates with a high level of accuracy, as compared to acase where the designated pixel real space coordinates are not specifiedon the basis of the imaging position distance, the designated pixelcoordinates, the focal length of the imaging lens, and the dimensions ofthe imaging pixel.

In the information processing device of a fourth aspect of the inventionaccording to the information processing device of any one of the firstto third aspects of the invention, the derivation unit derives adirection of a plane, including coordinates on the real space whichcorrespond to the plurality of pixel coordinates, which is specified bya plane equation indicating the plane on the basis of the plurality ofpixel coordinates, the focal length, and the dimensions, decides theplane equation on the basis of the derived direction and the emissionposition coordinates, and derives the imaging position distance on thebasis of the decided plane equation, the focal length, and thedimensions.

Therefore, according to the information processing device of the fourthaspect of the invention, it is possible to derive the imaging positiondistance with a high level of accuracy, as compared to a case where theimaging position distance is derived without using the plane equation ina case where a characteristic location capable of being specified is notirradiated with a laser beam.

In the information processing device according to a fifth aspect of theinvention according to the information processing device of any one ofthe first to fourth aspects, the plurality of pixels are designated byfirst pixel designation information for designating a pixel from each ofthe first captured image and the second captured image, the first pixeldesignation information being received by a first reception unitreceiving the first pixel designation information, the acquisition unitacquires a plurality of coordinates for specifying the plurality ofpixels designated in accordance with the first pixel designationinformation as the plurality of pixel coordinates, and the derivationunit derives the imaging position distance on the basis of the pluralityof pixel coordinates acquired by the acquisition unit, the emissionposition coordinates, the focal length, and the dimensions.

Therefore, according to the information processing device of the fifthaspect of the invention, it is possible to derive the imaging positiondistance on the basis of a plurality of pixel coordinates acquired by auser's intention.

In the information processing device of a sixth aspect of the inventionaccording to the information processing device of any one of the firstto fourth aspects of the invention, the acquisition unit acquires aplurality of coordinates, as the plurality of pixel coordinates, forspecifying a plurality of characteristic pixels more than three pixelswhich are present in the same planar region as the emission position onthe real space and correspond to the position on the real space in eachof the first captured image and the second captured image, and thederivation unit derives the imaging position distance on the basis ofthe plurality of pixel coordinates acquired by the acquisition unit, theemission position coordinates, the focal length, and the dimensions.

Therefore, according to the information processing device of the sixthaspect of the invention, it is possible to derive the imaging positiondistance on the basis of a plurality of pixel coordinates with a smallnumber of operations, as compared to a case where a plurality of pixelsfor specifying the plurality of pixel coordinates are designated by theuser in acquiring the plurality of pixel coordinates used for thederivation of the imaging position distance.

In the information processing device of a seventh aspect of theinvention according to the information processing device of the sixthaspect of the invention, the plurality of characteristic pixels are apredetermined number of pixels more than three pixels which are presentin the same planar region as the emission position on the real space andcorrespond to the position on the real space in each of the firstcaptured image and the second captured image, and are a plurality ofpixels for maximizing an area surrounded.

Therefore, according to the information processing device of the seventhaspect of the invention, it is possible to derive the imaging positiondistance with a high level of accuracy, as compared to a case where aplurality of pixels not for maximizing an area surrounded are adopted asthe plurality of characteristic pixels.

The information processing device of an eighth aspect of the inventionaccording to the information processing device of the sixth aspect ofthe invention further includes a first control unit that performscontrol of displaying at least one of the first captured image and thesecond captured image on a first display unit, and displaying acorresponding region corresponding to the same planar region as theemission position within a display region so as to be distinguishablefrom the other regions, in which the acquisition unit acquires aplurality of coordinates for specifying the plurality of characteristicpixels as the plurality of pixel coordinates, from a portion of thecorresponding region designated in accordance with region designationinformation received by a second reception unit receiving the regiondesignation information for designating a portion of the correspondingregion in a state where the corresponding region is displayed on thefirst display unit.

Therefore, according to the information processing device of the eighthaspect of the invention, it is possible to acquire a plurality of pixelcoordinates with a small load, as compared to a case where the pluralityof pixel coordinates are acquired from the entire corresponding region.

In the information processing device of a ninth aspect of the inventionaccording to the information processing device of any one of the firstto eighth aspects of the invention, the designated pixel, which isrelated to one of the first captured image and the second captured imageamong the designated pixels designated as a pixel corresponding to theposition on the real space in each of the first captured image and thesecond captured image which are acquired by the acquisition unit, is apixel designated in accordance with second pixel designation informationreceived by a third reception unit receiving the second pixeldesignation information for designating a pixel from one of the firstcaptured image and the second captured image, and the designated pixelrelated to the other one of the first captured image and the secondcaptured image is a pixel which is included in the other one of thefirst captured image and the second captured image and corresponds to aposition of the pixel designated in accordance with the second pixeldesignation information on the real space.

Therefore, according to the information processing device of the ninthaspect of the invention, it is possible to rapidly determine adesignated pixel related to both the first captured image and the secondcaptured image, as compared to a case where the designated pixel relatedto both the first captured image and the second captured image isdesignated by the user.

The information processing device of a tenth aspect of the inventionaccording to the information processing device of any one of the firstto ninth aspects of the invention further includes a measurement unitthat measures the distance by emitting the directional light andreceiving the reflected light, in which the acquisition unit acquiresthe distance measured by the measurement unit.

Therefore, according to the information processing device of the tenthaspect of the invention, it is possible to easily acquire the distanceused for the derivation of the emission position coordinates, ascompared to a case where the measurement unit is not provided.

The information processing device of an eleventh aspect of the inventionaccording to the information processing device of any one of the firstto tenth aspects of the invention further includes an imaging unit thatimages the subject, in which the acquisition unit that acquires thefirst captured image obtained by imaging the subject by the imaging unitfrom the first imaging position, and the second captured image obtainedby imaging the subject by the imaging unit from the second imagingposition.

Therefore, according to the information processing device of theeleventh aspect of the invention, it is possible to easily acquire thefirst captured image and the second captured image which are used toobtain the designated pixel coordinates and the plurality of pixelcoordinates, as compared to a case where the imaging unit is notprovided.

In the information processing device of a twelfth aspect of theinvention according to the information processing device of any one ofthe first to eleventh aspects of the invention, the acquisition unitfurther acquires a reference distance to the subject which is measuredby emitting the directional light to the subject from the other one ofthe position corresponding to the first imaging position and theposition corresponding to the second imaging position and receiving thereflected light of the directional light, and the derivation unitfurther derives a reference imaging position distance which is thedistance between the first imaging position and the second imagingposition on the basis of the plurality of pixel coordinates, referenceemission position coordinates for specifying the emission position onthe real space and derived on the basis of the reference distanceacquired by the acquisition unit, the focal length, and the dimensions,and adjusts the imaging position distance with reference to the derivedreference imaging position distance to derive a final imaging positiondistance which is finally adopted as the distance between the firstimaging position and the second imaging position.

Therefore, according to the information processing device of the twelfthaspect of the invention, it is possible to derive the distance betweenthe first imaging position and the second imaging position with a highlevel of accuracy, as compared to a case where the reference imagingposition distance is not used.

In the information processing device of a thirteenth aspect of theinvention according to the information processing device of the twelfthaspect of the invention, the derivation unit derives final designatedpixel real space coordinates, which are finally adopted as thecoordinates of the designated pixels on the real space which aredesignated as pixels corresponding to the position on the real space ineach of the first captured image and the second captured image which areacquired by the acquisition unit, on the basis of the derived finalimaging position distance.

Therefore, according to the information processing device of thethirteenth aspect of the invention, it is possible to derive thecoordinates of the designated pixel on the real space with a high levelof accuracy, as compared to a case where the coordinates of thedesignated pixel on the real space are derived without using the finalimaging position distance derived with reference to the referenceimaging position distance.

In the information processing device of a fourteenth aspect of theinvention according to the information processing device of thethirteenth aspect of the invention, the final designated pixel realspace coordinates are specified on the basis of the final imagingposition distance, the focal length, and the dimensions.

Therefore, according to the information processing device of thefourteenth aspect of the invention, it is possible to derive the finaldesignated pixel real space coordinates with a high level of accuracy,as compared to a case where the final designated pixel real spacecoordinates are not specified on the basis of the final imaging positiondistance, the designated pixel coordinates, the focal length of theimaging lens, and the dimensions of the imaging pixel.

The information processing device of a fifteenth aspect of the inventionaccording to the information processing device of any one of the firstto fourteenth aspects of the invention further includes a second controlunit that performs control of displaying derivation results of thederivation unit on a second display unit.

Therefore, according to the information processing device of thefifteenth aspect of the invention, it is possible to make the usereasily recognize the derivation results of the derivation unit, ascompared to a case where the derivation results of the derivation unitare not displayed.

An information processing method of a sixteenth aspect of the inventionincludes acquiring a first captured image obtained by imaging a subjectfrom a first imaging position, a second captured image obtained byimaging the subject from a second imaging position different from thefirst imaging position, and a distance from one of a positioncorresponding to the first imaging position and a position correspondingto the second imaging position to the subject, the distance beingmeasured by emitting directional light, which has directivity, to thesubject and receiving a reflected light of the directional light, andderiving an imaging position distance which is a distance between thefirst imaging position and the second imaging position, on the basis ofa plurality of pixel coordinates being a plurality of coordinates forspecifying a plurality of pixels of more than three pixels which arepresent in the same planar region as an emission position irradiatedwith the directional light on the real space and correspond to theposition on the real space in each of the acquired first captured imageand second captured image, emission position coordinates which specifiesthe emission position on the real space and are derived on the basis ofthe acquired distance, a focal length of an imaging lens used for theimaging of the subject, and dimensions of imaging pixels included in animaging pixel group for imaging the subject.

Therefore, according to the information processing method of thesixteenth aspect of the invention, it is possible to derive the imagingposition distance on the basis of captured images obtained by imagingthe subject from each of different imaging positions even when acharacteristic location capable of being specified is not irradiatedwith a laser beam.

A program of a seventeenth aspect of the invention, the program causinga computer to execute processes of acquiring a first captured imageobtained by imaging a subject from a first imaging position, a secondcaptured image obtained by imaging the subject from a second imagingposition different from the first imaging position, and a distance fromone of a position corresponding to the first imaging position and aposition corresponding to the second imaging position to the subject,the distance being measured by emitting directional light, which hasdirectivity, to the subject and receiving a reflected light of thedirectional light, and deriving an imaging position distance which is adistance between the first imaging position and the second imagingposition, on the basis of a plurality of pixel coordinates being aplurality of coordinates for specifying a plurality of pixels of morethan three pixels which are present in the same planar region as anemission position irradiated with the directional light on the realspace and correspond to the position on the real space in each of theacquired first captured image and second captured image, emissionposition coordinates which specifies the emission position on the realspace and are derived on the basis of the acquired distance, a focallength of an imaging lens used for the imaging of the subject, anddimensions of imaging pixels included in an imaging pixel group forimaging the subject.

Therefore, according to the program of the seventeenth aspect of theinvention, it is possible to derive the imaging position distance on thebasis of captured images obtained by imaging the subject from each ofdifferent imaging positions even when a characteristic location capableof being specified is not irradiated with a laser beam.

According to one embodiment of the invention, it is possible to obtainan effect that an imaging position distance can be derived on the basisof captured images obtained by performing imaging from each of differentimaging positions even when a characteristic location capable of beingspecified is not irradiated with a laser beam.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments according to the technique of the presentdisclosure will be described in detail based on the following figures,wherein:

FIG. 1 is a front view illustrating an example of the appearance of adistance measurement device according to first to fifth embodiments;

FIG. 2 is a block diagram illustrating an example of a hardwareconfiguration of the distance measurement device according to the firstto fourth embodiments;

FIG. 3 is a time chart illustrating an example of a measurement sequenceof the distance measurement device according to the first to sixthembodiments;

FIG. 4 is a time chart illustrating an example of a laser trigger, alight emission signal, a light receiving signal, and a count signalwhich are required in a case where measurement is performed once by thedistance measurement device according to the first to sixth embodiments;

FIG. 5 is a graph illustrating an example of a histogram (histogram in acase where a distance (measured value) to a subject is represented by alateral axis and the number of times of measurement is represented by avertical axis) of measured values obtained by the measurement sequenceof the distance measurement device according to the first to sixthembodiments;

FIG. 6 is a block diagram illustrating an example of a hardwareconfiguration of a main control unit included in the distancemeasurement device according to the first to sixth embodiments;

FIG. 7 is a schematic plan view illustrating an example of a positionalrelationship between the distance measurement device and the subjectaccording to the first to fourth embodiments and the sixth embodiment;

FIG. 8 is a conceptual diagram illustrating an example of a positionalrelationship between a portion of the subject, a first captured image, asecond captured image, a principal point of an imaging lens at a firstimaging position, and a principal point of the imaging lens at a secondimaging position;

FIG. 9 is a block diagram illustrating an example of a main function ofa CPU according to the first to sixth embodiments;

FIG. 10 is a diagram illustrating a method of calculating emissionposition coordinates according to the first to sixth embodiments;

FIG. 11 is a flowchart illustrating an example of a flow of an imagingposition distance calculation process according to the first embodiment;

FIG. 12 is the continuation of the flowchart illustrated in FIG. 11;

FIG. 13 is a conceptual diagram illustrating an example of a subjectincluded in an imaging range of an imaging device according to the firstto sixth embodiments;

FIG. 14 is a schematic image illustrating an example of the firstcaptured image obtained by the imaging device according to the firstembodiment;

FIG. 15 is a schematic image illustrating an example of the firstcaptured image obtained by the imaging device according to the firstembodiment;

FIG. 16 is a schematic image illustrating an example of the firstcaptured image obtained by the imaging device according to the firstembodiment;

FIG. 17 is a schematic image illustrating an example of the firstcaptured image obtained by the imaging device according to the firstembodiment;

FIG. 18 is a flowchart illustrating an example of a flow of athree-dimensional coordinate calculation process according to the firstembodiment;

FIG. 19 is a schematic image illustrating an example of the secondcaptured image obtained by the imaging device according to the firstembodiment;

FIG. 20 is a schematic image illustrating an example of the firstcaptured image obtained by the imaging device according to the firstembodiment;

FIG. 21 is a flowchart illustrating an example of a flow of an imagingposition distance calculation process according to the secondembodiment;

FIG. 22 is a schematic image illustrating an example of a first capturedimage obtained by an imaging device according to the second embodiment;

FIG. 23 is a schematic image illustrating an example of the firstcaptured image obtained by the imaging device according to the secondembodiment;

FIG. 24 is a flowchart illustrating an example of a flow of an imagingposition distance calculation process according to the third embodiment;

FIG. 25 is the continuation of the flowchart illustrated in FIG. 24;

FIG. 26 is a flowchart illustrating an example of a flow of an imagingposition distance calculation process according to the fourthembodiment;

FIG. 27 is a schematic image illustrating an example of a secondcaptured image obtained by an imaging device according to the fourthembodiment;

FIG. 28 is a flowchart illustrating an example of a flow of athree-dimensional coordinate calculation process according to the fourthembodiment;

FIG. 29 is a schematic image illustrating an example of the secondcaptured image obtained by the imaging device according to the fourthembodiment;

FIG. 30 is a schematic plan view illustrating an example of a positionalrelationship between two distance measurement devices, a PC, and asubject according to the fifth embodiment;

FIG. 31 is a block diagram illustrating an example of a hardwareconfiguration of a distance measurement device according to the fifthembodiment;

FIG. 32 is a block diagram illustrating an example of a hardwareconfiguration of the PC according to the fifth embodiment;

FIG. 33 is a block diagram illustrating an example of a hardwareconfiguration of a distance measurement device according to the sixthembodiment;

FIG. 34 is a screen view illustrating an example of a screen includingvarious buttons displayed as soft keys on a display unit of a smartdevice included in the distance measurement device according to thesixth embodiment;

FIG. 35 is a conceptual diagram illustrating an example of a mode inwhich an imaging position distance calculation program and athree-dimensional coordinate calculation program are installed in thedistance measurement device or the PC from a storage medium in which theimaging position distance calculation program and the three-dimensionalcoordinate calculation program according to the first to fourthembodiments are stored; and

FIG. 36 is a front view illustrating a modification example of theappearance of the distance measurement device according to the first tosixth embodiments.

DETAILED DESCRIPTION

Hereinafter, an example of an embodiment according to a technique ofthis disclosure will be described with reference to the accompanyingdrawings. Meanwhile, in this embodiment, for convenience of description,a distance from a distance measurement device 10A to a subject to bemeasured will be also simply referred to as a “distance” or a “distanceto a subject”. In this embodiment, an angle of view with respect to asubject will be also simply referred to as an “angle of view”.

First Embodiment

As illustrated in FIG. 1 as an example, the distance measurement device10A which is an example of an information processing device according tothe technique of this disclosure includes a distance measurement unit 12and an imaging device 14. Meanwhile, in this embodiment, the distancemeasurement unit 12 and a distance measurement control unit 68 to bedescribed later (see FIG. 2) are examples of a measurement unitaccording to the technique of this disclosure, and the imaging device 14is an example of an imaging unit according to the technique of thisdisclosure.

The imaging device 14 includes a lens unit 16 and an imaging device mainbody 18, and the lens unit 16 is detachably attached to the imagingdevice main body 18.

A hot shoe (Hot Shoe) 20 is provided on the left surface of the imagingdevice main body 18 in a front view, and the distance measurement unit12 is detachably attached to the hot shoe 20.

The distance measurement device 10A has a distance measurement systemfunction of emitting a laser beam for distance measurement to thedistance measurement unit 12 to perform distance measurement and animaging system function of causing the imaging device 14 to image asubject to obtain a captured image. Meanwhile, hereinafter, a capturedimage will be also simply referred to as an “image”. In addition,hereinafter, for convenience of description, a description will be givenon the assumption that the height of an optical axis L1 (see FIG. 2) ofa laser beam emitted from the distance measurement unit 12 is the sameas the height of an optical axis L2 (see FIG. 2) of the lens unit 16 inthe vertical direction.

The distance measurement device 10A operates the distance measurementsystem function to perform a measurement sequence (see FIG. 3) once inaccordance with one instruction, and one distance is finally output bythe measurement sequence being performed once,

The distance measurement device 10A has a still image imaging mode and amovie imaging mode as an operation mode of the imaging system function.The still image imaging mode is an operation mode for capturing a stillimage, and the movie imaging mode is an operation mode for capturing amoving image. The still image imaging mode and the movie imaging modeare selectively set in accordance with a user's instruction.

As illustrated in FIG. 2 as an example, the distance measurement unit 12includes an emitting unit 22, a light receiving unit 24, and a connector26.

The connector 26 can be connected to the hot shoe 20, and the distancemeasurement unit 12 is operated under the control of the imaging devicemain body 18 in a state where the connector 26 is connected to the hotshoe 20.

The emitting unit 22 includes a Laser Diode (LD) 30, a condensing lens(not shown), an objective lens 32, and an LD driver 34.

The condensing lens and the objective lens 32 are provided along theoptical axis L1 of a laser beam emitted by the LD 30, and are disposedin this order along the optical axis L1 from the LD 30 side.

The LD 30 emits a laser beam for distance measurement which is anexample of a directional light according to the technique of thisdisclosure. The laser beam emitted by the LD 30 is a colored laser beam,and a real space emission position which is an emission position on thereal space of the laser beam is visually recognized on the real spaceand is also visually recognized from a captured image obtained by theimaging device 14, for example, within a range of approximately severalmeters from the emitting unit 22. Meanwhile, hereinafter, forconvenience of description, the real space emission position of thelaser beam will also be simply referred to as an “emission position”.

The condensing lens condenses a laser beam emitted by the LD 30, andtransmits the condensed laser beam. The objective lens 32 faces asubject, and emits the laser beam passing through the condensing lens tothe subject.

The LD driver 34 is connected to the connector 26 and the LD 30, anddrives the LD 30 in accordance with an instruction of the imaging devicemain body 18 to emit a laser beam.

The light receiving unit 24 includes a Photo Diode (PD) 36, an objectivelens 38, and a light receiving signal processing circuit 40. Theobjective lens 38 is disposed on a light receiving surface side of thePD 36, and a reflected laser beam which is a laser beam emitted by theemitting unit 22 and reflected from the subject is incident on theobjective lens 38. The objective lens 38 transmits the reflected laserbeam and guides the reflected laser beam to the light receiving surfaceof the PD 36. The PD 36 receives the reflected laser beam having passedthrough the objective lens 38, and outputs an analog signal based on theamount of light received, as a light receiving signal.

The light receiving signal processing circuit 40 is connected to theconnector 26 and the PD 36, amplifies the light receiving signal, whichis input from the PD 36, by an amplifier (not shown), and performsAnalog/Digital (A/D) conversion on the amplified light receiving signal.The light receiving signal processing circuit 40 outputs the lightreceiving signal digitalized by the A/D conversion to the imaging devicemain body 18.

The imaging device 14 includes mounts 42 and 44. The mount 42 isprovided in the imaging device main body 18, and the mount 44 isprovided in the lens unit 16. The lens unit 16 is exchangeably mountedon the imaging device main body 18 by the mount 44 being coupled to themount 42.

The lens unit 16 includes an imaging lens 50, a zoom lens 52, a zoomlens moving mechanism 54, and a motor 56.

Subject light which is light reflected from the subject is incident onthe imaging lens 50. The imaging lens 50 transmits the subject light andguides the subject light to the zoom lens 52.

The zoom lens 52 is attached to the zoom lens moving mechanism 54 so asto be slidable with respect to the optical axis L2. In addition, themotor 56 is connected to the zoom lens moving mechanism 54, and the zoomlens moving mechanism 54 receives the power of the motor 56 to make thezoom lens 52 slide along the direction of the optical axis L2.

The motor 56 is connected to the imaging device main body 18 through themounts 42 and 44, and driving is controlled in accordance with a commandfrom the imaging device main body 18. Meanwhile, in this embodiment, astepping motor is applied as an example of the motor 56. Therefore, themotor 56 is operated in synchronization with a pulse power on the basisof a command from the imaging device main body 18.

The imaging device main body 18 includes an imaging element 60, a maincontrol unit 62, an image memory 64, an image processing unit 66, adistance measurement control unit 68, a motor driver 72, an imagingelement driver 74, an image signal processing circuit 76, and a displaycontrol unit 78. In addition, the imaging device main body 18 includes atouch panel interface (I/F) 79, a reception I/F 80, and a media I/F 82.

The main control unit 62, the image memory 64, the image processing unit66, the distance measurement control unit 68, the motor driver 72, theimaging element driver 74, the image signal processing circuit 76, andthe display control unit 78 are connected to a bus line 84. In addition,the touch panel I/F 79, the reception I/F 80, and the media I/F 82 arealso connected to the bus line 84.

The imaging element 60 is a Complementary Metal Oxide Semiconductor(CMOS) type image sensor, and includes color filters (not shown). Thecolor filters include a G filter corresponding to green (G), an R filtercorresponding to red (R), and a B filter corresponding to blue (B) whichmost contribute to the obtainment of a brightness signal. The imagingelement 60 includes an imaging pixel group 60A including a plurality ofimaging pixels 60A1 arranged in a matrix. Any one filter of the Rfilter, the G filter, and the B filter included in the color filters isallocated to each of the imaging pixels 60A1, and the imaging pixelgroup 60A receives the subject light to image the subject.

That is, the subject light having passed through the zoom lens 52 isimaged on the light receiving surface of the imaging element 60, andcharge based on the amount of subject light received is accumulated inthe imaging pixels 60A1. The imaging element 60 outputs the chargeaccumulated in the imaging pixels 60A1 as an image signal indicating animage equivalent to a subject image which is obtained by imaging thesubject light on the light receiving surface.

The main control unit 62 controls the entire distance measurement device10A through the bus line 84.

The motor driver 72 is connected to the motor 56 through the mounts 42and 44, and controls the motor 56 in accordance with an instruction ofthe main control unit 62.

The imaging device 14 has a viewing angle changing function. The viewingangle changing function is a function of changing an angle of view bymoving the zoom lens 52, and is realized by the zoom lens 52, the zoomlens moving mechanism 54, the motor 56, the motor driver 72, and themain control unit 62 in this embodiment. Meanwhile, in this embodiment,an optical viewing angle changing function of the zoom lens 52 isdescribed. However, the technique of this disclosure is not limitedthereto, an electronic viewing angle changing function not using thezoom lens 52 may be used.

The imaging element driver 74 is connected to the imaging element 60,and provides a driving pulse to the imaging element 60 under the controlof the main control unit 62. The imaging pixels 60A1 included in theimaging pixel group 60A are driven in accordance with the driving pulsesupplied to the imaging element 60 by the imaging element driver 74.

The image signal processing circuit 76 is connected to the imagingelement 60, and reads out an image signal for one frame from the imagingelement 60 for each imaging pixel 60A1 under the control of the maincontrol unit 62. The image signal processing circuit 76 performs variousprocessing, such as correlative double sampling processing, automaticgain control, and A/D conversion, on the read-out image signal. Theimage signal processing circuit 76 outputs an image signal, which isdigitalized by performing various processing on the image signal, to theimage memory 64 for each frame at a specific frame rate (for example,several tens of frames per second) which is specified by a clock signalsupplied from the main control unit 62. The image memory 64 temporarilyholds the image signal which is input from the image signal processingcircuit 76.

The imaging device main body 18 includes a display unit 86, a touchpanel 88, a reception device 90, and a memory card 92.

The display unit 86 which is an example of each of a first display unitand a second display unit according to the technique of this disclosureis connected to the display control unit 78, and displays variousinformation under the control of the display control unit 78. Thedisplay unit 86 is realized by, for example, a Liquid Crystal Display(LCD).

The touch panel 88 which is an example of each of first to thirdreception units according to the technique of this disclosure issuperimposed on a display screen of the display unit 86, and receives atouch of a user's finger or an indicator such as a touch pen. The touchpanel 88 is connected to the touch panel I/F 79, and outputs positionalinformation indicating a position touched by the indicator to the touchpanel I/F 79. The touch panel I/F 79 operates the touch panel 88 inaccordance with an instruction of the main control unit 62, and outputsthe positional information, which is input from the touch panel 88, tothe main control unit 62. Meanwhile, in this embodiment, the touch panel88 is described as an example of the first to third reception unitsaccording to the technique of this disclosure, but the invention is notlimited thereto. A mouse (not shown) used by being connected to thedistance measurement device 10A may be applied instead of the touchpanel 88, or the touch panel 88 and the mouse may be used incombination.

The reception device 90 includes a measurement and imaging button 90A,an imaging button 90B, an imaging system operation mode switching button90C, a wide angle instruction button 90D, and a telephoto instructionbutton 90E. In addition, the reception device 90 also includes animaging position distance calculation button 90F, a three-dimensionalcoordinate calculation button 90G, and the like, and receives the user'svarious instructions. The reception device 90 is connected to thereception I/F 80, and the reception I/F 80 outputs an instructioncontent signal indicating contents of an instruction received by thereception device 90 to the main control unit 62.

The measurement and imaging button 90A is a pressing type button thatreceives an instruction for starting measurement and imaging. Theimaging button 90B is a pressing type button that receives aninstruction for starting imaging. The imaging system operation modeswitching button 90C is a pressing type button that receives aninstruction for switching between a still image imaging mode and a movieimaging mode.

The wide angle instruction button 90D is a pressing type button thatreceives an instruction for setting an angle of view to be a wide angle,and the amount of change of the angle of view to the wide angle side isdetermined depending on a pressing time for which the pressing of thewide angle instruction button 90D is continuously performed within anallowable range.

The telephoto instruction button 90E is a pressing type button thatreceives an instruction for setting an angle of view to be at atelephoto side, the amount of change of the angle of view to thetelephoto side is determined depending on a pressing time for which thepressing of the telephoto instruction button 90E is continuouslyperformed within an allowable range.

The imaging position distance calculation button 90F is a pressing typebutton that receives an instruction for starting an imaging positiondistance calculation process to be described later. Thethree-dimensional coordinate calculation button 90G is a pressing typebutton that receives an instruction for starting an imaging positiondistance calculation process to be described later and athree-dimensional coordinate calculation process to be described later.

Meanwhile, hereinafter, for convenience of description, the measurementand imaging button 90A and the imaging button 90B will be referred to asa “release button” in a case where it is not necessary to give adescription by distinguishing between the buttons. In addition,hereinafter, for convenience of description, the wide angle instructionbutton 90D and the telephoto instruction button 90E will be referred toas an “angle of view instruction button” in a case where it is notnecessary to give a description by distinguishing between the buttons.

Meanwhile, in the distance measurement device 10A according to thisembodiment, a manual focus mode and an autofocus mode are selectivelyset in accordance with the user's instruction through the receptiondevice 90. The release button receives two-stage pressing operations ofan imaging preparation instruction state and an imaging instructionstate. The imaging preparation instruction state refers to, for example,a state where the release button is pressed to an intermediate position(half pressing position) from a waiting position, and the imaginginstruction state refers to a state where the release button is pressedto a final pressing position (full pressing position) beyond theintermediate position. Meanwhile, hereinafter, for convenience ofdescription, the “state where the release button is pressed to the halfpressing position from the waiting position” will be referred to as a“half pressing state”, and the “state where the release button ispressed to the full pressing position from the waiting position” will bereferred to as a “full pressing state”.

In the autofocus mode, the adjustment of imaging conditions is performedby the release button being set to be in a half pressing state.Thereafter, when the release button is subsequently set to be in a fullpressing state, the actual exposure is performed. That is, afterexposure adjustment is performed by the operation of an AutomaticExposure (AE) function by the release button being set to be in a halfpressing state prior to the actual exposure, focus adjustment isperformed by the operation of an Auto-Focus (AF) function, and theactual exposure is performed when the release button is set to be in afull pressing state.

Here, the actual exposure refers to exposure performed to obtain a stillimage file to be described later. In this embodiment, the exposure meansexposure performed to obtain a live view image to be described later andexposure performed to obtain a moving image file to be described later,in addition to the actual exposure. Hereinafter, for convenience ofdescription, the exposures will be simply referred to as “exposure” in acase where it is not necessary to give a description by distinguishingbetween the exposures.

Meanwhile, in this embodiment, the main control unit 62 performsexposure adjustment based on an AE function and focus adjustment basedon an AF function. In this embodiment, a case where the exposureadjustment and the focus adjustment are performed is described. However,the technique of this disclosure is not limited thereto, and theexposure adjustment or the focus adjustment may be performed.

The image processing unit 66 acquires an image signal for each framefrom the image memory 64 at a specific frame rate, and performs variousprocessing, such as gamma correction, brightness and color differenceconversion, and compression processing, on the acquired image signal.

The image processing unit 66 outputs the image signal, which is obtainedby performing various processing, to the display control unit 78 foreach frame at a specific frame rate. In addition, the image processingunit 66 outputs the image signal, which is obtained by performingvarious processing, to the main control unit 62 in accordance with arequest of the main control unit 62.

The display control unit 78 outputs the image signal, which is inputfrom the image processing unit 66, to the display unit 86 for each frameat a specific frame rate under the control of the main control unit 62.

The display unit 86 displays an image, character information, and thelike. The display unit 86 displays an image shown by the image signal,which is input from the display control unit 78 at a specific framerate, as a live view image. The live view image is a consecutive frameimage which is obtained by consecutive imaging, and is also referred toas a through-image. In addition, the display unit 86 also displays astill image which is a single frame image obtained by performing imagingusing a single frame. Further, the display unit 86 also displays areproduced image, a menu screen, and the like, in addition to the liveview image.

Meanwhile, in this embodiment, the image processing unit 66 and thedisplay control unit 78 are realized by an Application SpecificIntegrated Circuit (ASIC), but the technique of this disclosure is notlimited thereto. For example, each of the image processing unit 66 andthe display control unit 78 may be realized by a Field-Programmable GateArray (FPGA). In addition, the image processing unit 66 may be realizedby a computer including a Central Processing Unit (CPU), a Read OnlyMemory (ROM), and a Random Access Memory (RAM). In addition, the displaycontrol unit 78 may also be realized by a computer including a CPU, aROM, and a RAM. Further, each of the image processing unit 66 and thedisplay control unit 78 may be realized by a combination of a hardwareconfiguration and a software configuration.

The main control unit 62 controls the imaging element driver 74 to causethe imaging element 60 to perform exposure for each frame in a casewhere an instruction for capturing a still image is received by therelease button under a still image imaging mode. The main control unit62 acquires an image signal, which is obtained by performing theexposure for each frame, from the image processing unit 66 and performscompression processing on the acquired image signal to generate a stillimage file having a specific still image format. Meanwhile, here, thespecific still image format refers to, for example, Joint PhotographicExperts Group (JPEG).

The main control unit 62 acquires an image signal, which is output tothe display control unit 78 as a signal for a live view image by theimage processing unit 66, for each frame at a specific frame rate in acase where an instruction for capturing a moving image is received bythe release button under a movie imaging mode. The main control unit 62performs compression processing on the image signal acquired from theimage processing unit 66 to generate a moving image file having aspecific moving image format. Meanwhile, here, the specific moving imageformat refers to, for example, Moving Picture Experts Group (MPEG).Meanwhile, hereinafter, for convenience of description, the still imagefile and the moving image file will be referred to as an image file in acase where it is not necessary to give a description by distinguishingbetween the image files.

The media I/F 82 is connected to the memory card 92, and performs therecording and read-out of the image file on the memory card 92 under thecontrol of the main control unit 62. Meanwhile, the image file which isread out from the memory card 92 by the media I/F 82 is subjected toextension processing by the main control unit 62 to be displayed on thedisplay unit 86 as a reproduced image.

Meanwhile, the main control unit 62 stores distance information, whichis input from the distance measurement control unit 68, in the memorycard 92 through the media I/F 82 in association with the image file. Thedistance information is read out together with the image file by themain control unit 62 from the memory card 92 through the media I/F 82,and a distance indicated by the read-out distance information isdisplayed on the display unit 86 together with the reproduced imagebased on the associated image file.

The distance measurement control unit 68 controls the distancemeasurement unit 12 under the control of the main control unit 62.Meanwhile, in this embodiment, the distance measurement control unit 68is realized by an ASIC, but the technique of this disclosure is notlimited thereto. For example, the distance measurement control unit 68may be realized by a FPGA. In addition, the distance measurement controlunit 68 may be realized by a computer including a CPU, a ROM, and a RAM.Further, the distance measurement control unit 68 may be realized by acombination of a hardware configuration and a software configuration.

The hot shoe 20 is connected to the bus line 84, and the distancemeasurement control unit 68 controls the LD driver 34 to control theemission of a laser beam by the LD 30 under the control of the maincontrol unit 62 and acquires a light receiving signal from the lightreceiving signal processing circuit 40. The distance measurement controlunit 68 derives a distance to the subject on the basis of a timing whenthe laser beam is emitted and a timing when the light receiving signalis acquired, and outputs distance information indicating the deriveddistance to the main control unit 62.

Here, the measurement of a distance to the subject by the distancemeasurement control unit 68 will be described in more detail.

As illustrated in FIG. 3 as an example, one measurement sequence by thedistance measurement device 10A is specified by a voltage adjustmentperiod, a real measurement period, and a pause period.

The voltage adjustment period is a period in which driving voltages ofthe LD 30 and the PD 36 are adjusted. The real measurement period is aperiod in which a distance to the subject is actually measured. In thereal measurement period, an operation of causing the LD 30 to emit alaser beam and causing the PD 36 to receive the reflected laser beam isrepeated several hundred times, and a distance to the subject is derivedon the basis of a timing when the laser beam is emitted and a timingwhen the light receiving signal is acquired. The pause period is aperiod for stopping the driving of the LD 30 and the PD 36. Accordingly,in one measurement sequence, the measurement of a distance to thesubject is performed several hundred times.

Meanwhile, in this embodiment, each of the voltage adjustment period,the real measurement period, and the pause period is set to be severalhundred milliseconds.

As illustrated in FIG. 4 as an example, a count signal for specifying atiming when the distance measurement control unit 68 gives aninstruction for emitting a laser beam and a timing when a lightreceiving signal is acquired is provided to the distance measurementcontrol unit 68. In this embodiment, the count signal is generated bythe main control unit 62 and is supplied to the distance measurementcontrol unit 68. However, the invention is not limited thereto, and thecontrol signal may be generated by a dedicated circuit, such as a timecounter, which is connected to the bus line 84, and may be supplied tothe distance measurement control unit 68.

The distance measurement control unit 68 outputs a laser trigger foremitting a laser beam to the LD driver 34 in accordance with the countsignal. The LD driver 34 drives the LD 30 to emit a laser beam inaccordance with the laser trigger.

In the example illustrated in FIG. 4, a light emission time of a laserbeam is set to be several tens of nanoseconds. In this case, a timeuntil the laser beam, which is emitted toward a subject positionedseveral kilometers ahead by the emitting unit 22, is received by the PD36 as a reflected laser beam is set to be “several kilometers×2/speed oflight”=several microseconds. Therefore, as illustrated in FIG. 3 as anexample, a time of several microseconds is required as a minimumnecessary time in order to measure a distance to the subject positionedseveral kilometers ahead.

Meanwhile, in this embodiment, as illustrated in FIG. 3 as an example,one measurement time is set to be several milliseconds in considerationof a reciprocating time of the laser beam, and the like. However, thereciprocating time of the laser beam varies depending on a distance tothe subject, and thus one measurement time may vary in accordance withan assumed distance.

In a case where a distance to the subject is derived on the basis ofmeasured values obtained from several hundred times of measurement inone measurement sequence, the distance measurement control unit 68analyzes, for example, a histogram of the measured values obtained fromseveral hundred times of measurement to derive a distance to thesubject.

As illustrated in FIG. 5 as an example, in a histogram of measuredvalues obtained from several hundred times of measurement in onemeasurement sequence, the lateral axis represents a distance to asubject, the vertical axis represents the number of times ofmeasurement, and a distance corresponding to a maximum value of thenumber of times of measurement is derived by the distance measurementcontrol unit 68 as a distance measurement result. Meanwhile, thehistogram illustrated in FIG. 5 is just an example, and a histogram maybe generated on the basis of a reciprocating time (an elapsed time fromthe emission of light to the reception of light) of a laser beam, halfof the reciprocating time of the laser beam, and the like, instead ofthe distance to the subject.

As illustrated in FIG. 6 as an example, the main control unit 62includes a CPU 100, a primary storage unit 102, and a secondary storageunit 104 which are examples of an acquisition unit and a derivation unitaccording to the technique of this disclosure. The CPU 100 controls theentire distance measurement device 10A. The primary storage unit 102 isa volatile memory which is used as a work area during the execution ofvarious programs, and the like. An example of the primary storage unit102 is a RAM. The secondary storage unit 104 is a non-volatile memorythat stores control programs, various parameters, and the like forcontrolling the operation of the distance measurement device 10A. Anexample of the secondary storage unit 104 is an Electrically ErasableProgrammable Read Only Memory (EEPROM) and a flash memory. The CPU 100,the primary storage unit 102, and the secondary storage unit 104 areconnected to each other through the bus line 84.

The distance measurement device 10A has a three-dimensional coordinatecalculation function. The three-dimensional coordinate calculationfunction refers to a function of calculating designated pixelthree-dimensional coordinates to be described later, on the basis ofExpression (1) from first designated pixel coordinates to be describedlater, second designated pixel coordinates to be described later, animaging position distance to be described later, a focal length of theimaging lens 50, and the dimension of the imaging pixel 60A1.

$\begin{matrix}{{X = {\frac{B}{u_{L} - u_{R}}u_{L}}},{Y = {\frac{B}{u_{L} - u_{R}}v_{L}}},{Z = {\frac{B}{u_{L} - u_{R}}f}}} & (1)\end{matrix}$

Meanwhile, in Expression (1), “u_(L)” denotes an X coordinate of thefirst designated pixel coordinates. In Expression (1), “v_(L)” denotes aY coordinate of the first designated pixel coordinates. In Expression(1), “u_(R)” denotes an X coordinate of the second designated pixelcoordinates. In Expression (1), “B” denotes the imaging positiondistance (see FIGS. 7 and 8). In Expression (1), “f” denotes (focallength of the imaging lens 50)/(dimension of the imaging pixel 60A1). InExpression (1), (X, Y, Z) denotes the designated pixel three-dimensionalcoordinates.

The first designated pixel coordinates are two-dimensional coordinatesfor specifying a first designated pixel (equivalent to a “designatedpixel” according to the technique of this disclosure) which isdesignated as a pixel corresponding to a position on the real space in afirst captured image to be described later. The second designated pixelcoordinates are two-dimensional coordinates for specifying a seconddesignated pixel (equivalent to a “designated pixel” according to thetechnique of this disclosure) which is designated as a pixelcorresponding to a position on the real space in a second captured imageto be described later. That is, the first designated pixel and thesecond designated pixel are pixels that are designated as pixels ofwhich the positions on the real space correspond to each other, and arepixels capable of being specified at the positions corresponding to eachother in each of the first captured image and the second captured image.The first designated pixel coordinates are two-dimensional coordinateson the first captured image, and the second designated pixel coordinatesare two-dimensional coordinates on the second captured image.

The designated pixel three-dimensional coordinates refer tothree-dimensional coordinates which are coordinates on the real spacewhich correspond to the first designated pixel coordinates and thesecond designated pixel coordinates. Meanwhile, the designated pixelthree-dimensional coordinates are an example of designated pixel realspace coordinates according to the technique of this disclosure.

Here, as illustrated in FIGS. 7 and 8 as examples, the first capturedimage refers to a captured image obtained by imaging the subject by theimaging device 14 from the first imaging position. In addition, as anexample, as illustrated in FIGS. 7 and 8, the second captured imageindicates a captured image obtained by imaging a subject, including thesubject imaged from the first imaging position, by the imaging device 14from the second imaging position different from the first imagingposition. Meanwhile, the invention is not limited to the first capturedimage and the second captured image. In this embodiment, for convenienceof description, captured images obtained by the imaging of the imagingdevice 14, inclusive of a still image and a moving image, will be simplyreferred to as a “captured image” in a case where it is not necessary togive a description by distinguishing between the captured images.

In the example illustrated in FIG. 7, a first measurement position and asecond measurement position are shown as positions of the distancemeasurement unit 12. The first measurement position is an example of a“position corresponding to the first imaging position” according to thetechnique of this disclosure. The second measurement position is anexample of a “position corresponding to the second imaging position”according to the technique of this disclosure. The first measurementposition indicates the position of the distance measurement unit 12 in acase where the subject is imaged by the imaging device 14 from the firstimaging position in a state where the distance measurement unit 12 iscorrectly attached to the imaging device 14. The second measurementposition refers to the position of the distance measurement unit 12 in acase where the subject is imaged by the imaging device 14 from thesecond imaging position in a state where the distance measurement unit12 is correctly attached to the imaging device 14.

The imaging position distance refers to a distance between the firstimaging position and the second imaging position. As illustrated in FIG.8, an example of the imaging position distance is a distance between aprincipal point O_(L) of the imaging lens 50 of the imaging device 14 atthe first imaging position and a principal point O_(R) of the imaginglens 50 of the imaging device 14 at the second imaging position, but thetechnique of this disclosure is not limited thereto. For example, adistance between the imaging pixel 60A1 positioned in the middle of theimaging element 60 of the imaging device 14 at the first imagingposition and the imaging pixel 60A1 positioned in the middle of theimaging element 60 of the imaging device 14 at the second imagingposition may be set to be an imaging position distance.

In the example illustrated in FIG. 8, a pixel P_(L) included in thefirst captured image is a first designated pixel, a pixel P_(R) includedin the second captured image is a second designated pixel, and pixelsP_(L) and P_(R) are pixels corresponding to a point P of the subject.Accordingly, first designated pixel coordinates (u_(L), v_(L)) which aretwo-dimensional coordinates of the pixel P_(L) and second designatedpixel coordinates (u_(R), v_(R)) which are two-dimensional coordinatesof the pixel P_(R) correspond to designated pixel three-dimensionalcoordinates (X, Y, Z) which are three-dimensional coordinates of thepoint P. Meanwhile, in Expression (1), “v_(R)” is not used.

Meanwhile, hereinafter, for convenience of description, the firstdesignated pixel and the second designated pixel will be referred to asa “designated pixel” in a case where it is not necessary to give adescription by distinguishing between the designated pixels. Further,hereinafter, for convenience of description, the first designated pixelcoordinates and the second designated pixel coordinates will be referredto as “designated pixel coordinates” in a case where it is not necessaryto give a description by distinguishing between the designated pixelcoordinates.

Incidentally, in a case where designated pixel three-dimensionalcoordinates are calculated on the basis of Expression (1) by thedistance measurement device 10A operating a three-dimensional coordinatecalculation function, it is preferable to calculate an imaging positiondistance with a high level of accuracy. This is because “B” which is animaging position distance is included in Expression (1).

Consequently, in the distance measurement device 10A, as illustrated inFIG. 6 as an example, the secondary storage unit 104 stores an imagingposition distance calculation program 106 which is an example of aprogram according to the technique of this disclosure.

The CPU 100 reads out the imaging position distance calculation program106 from the secondary storage unit 104 and develops the read-outprogram to the primary storage unit 102 to execute the imaging positiondistance calculation program 106.

In addition, as illustrated in FIG. 6 as an example, the secondarystorage unit 104 stores a three-dimensional coordinate calculationprogram 108. The CPU 100 reads out the three-dimensional coordinatecalculation program 108 from the secondary storage unit 104 and developsthe read-out program to the primary storage unit 102 to execute thethree-dimensional coordinate calculation program 108.

The CPU 100 executes the imaging position distance calculation program106 and the three-dimensional coordinate calculation program 108 to beoperated as an acquisition unit 110, a derivation unit 112, and acontrol unit 114 as illustrated in FIG. 9 as an example.

The acquisition unit 110 acquires a first captured image, a secondcaptured image, and a distance to the subject. The “distance to thesubject” as mentioned herein refers to a distance to the subject whichis measured on the basis of the laser beam emitted by the distancemeasurement unit 12 at the first measurement position.

The derivation unit 112 derives an imaging position distance on thebasis of designated pixel coordinates, a plurality of pixel coordinatesto be described later, emission position real space coordinates to bedescribed later, a focal length of the imaging lens 50, and thedimension of the imaging pixel 60A1. The control unit 114 which is anexample of a second control unit according to the technique of thisdisclosure performs control of displaying derivation results obtained bythe derivation unit 112 on the display unit 86.

Here, the plurality of pixel coordinates and the emission position realspace coordinates which are used in the derivation unit 112 will bedescribed. Meanwhile, hereinafter, for convenience of description, the“emission position real space coordinates” will also be referred to as“emission position coordinates”. The plurality of pixel coordinates area plurality of two-dimensional coordinates for specifying a plurality ofpixels more than three pixels which are present in a planar region whichis the same as an emission position of a laser beam on the real spaceand correspond to a position on the real space, in each of the firstcaptured image and the second captured image which are acquired by theacquisition unit 110. In addition, the emission position coordinates arethree-dimensional coordinates for specifying the emission position ofthe laser beam on the real space, and are three-dimensional coordinateswhich are derived on the basis of a distance acquired by the acquisitionunit 110.

The emission position coordinates are derived on the basis of thefollowing Expression (2) from a distance L, a half angle of view α, anemission angle β, and a distance between reference points M which areillustrated in FIG. 10 as an example. In Expression (2), (x_(Laser),y_(Laser), z_(Laser)) denotes emission position coordinates.

$\begin{matrix}{{x_{Laser} = \frac{\left( {M - {L\;\cos\;\beta}} \right)}{L\;\tan\;{\alpha sin\beta}}},{y_{Laser} = 0},{z_{Laser} = {L\;\sin\;\beta}}} & (2)\end{matrix}$

In Expression (2), the relation of y_(Laser)=0 is established, but thismeans that the height of an optical axis L1 is the same as the height ofan optical axis L2 in the vertical direction. In a case where theposition of a laser beam emitted to the subject is higher than theposition of the optical axis L2 in the subject in the verticaldirection, y_(Laser) is set to have a positive value. In a case wherethe position of the laser beam emitted to the subject is lower than theposition of the optical axis L2 in the subject in the verticaldirection, y_(Laser) is set to have a negative value. Meanwhile,hereinafter, for convenience of description, a description will be givenon the assumption that the relation of “y_(Laser)=0” is established.

Here, as illustrated in FIG. 10 as an example, the half angle of view αrefers to half an angle of view. The emission angle β refers to an angleat which a laser beam is emitted from the emitting unit 22. The distancebetween reference points M refers to a distance between a firstreference point P1 specified for the imaging device 14 and a secondreference point P2 specified for the distance measurement unit 12. Anexample of the first reference point P1 is a principal point of theimaging lens 50. An example of the second reference point P2 is a pointwhich is set in advance as the starting point of coordinates capable ofspecifying the position of a three-dimensional space in the distancemeasurement unit 12. Specifically, an example of the second referencepoint is one end out of right and left ends of the objective lens 38 ina front view, or one angle, that is, one apex of a housing in a casewhere the housing (not shown) of the distance measurement unit 12 has arectangular parallelepiped shape.

The derivation unit 112 derives the direction of a plane specified by aplane equation showing a plane including three-dimensional coordinateson the real space which correspond to a plurality of pixel coordinates,on the basis of the plurality of pixel coordinates, a focal length ofthe imaging lens 50, and the dimension of the imaging pixel 60A1. Thederivation unit 112 decides the plane equation on the basis of thederived direction of the plane and the emission position coordinates,and derives the an imaging position distance on the basis of the decidedplane equation, the designated pixel coordinates, the focal length ofthe imaging lens 50, and the dimension of the imaging pixel 60A1.

Meanwhile, the plane equation which is used for the derivation of theimaging position distance is specified by the following Expression (3).Therefore, the derivation of the “direction of the plane” means that a,b, and c in Expression (3) are derived, and the fixedly deriving of the“plane equation” means that d in Expression (3) is derived to fixedlyderive a, b, c, and d of the plane equation.ax+by+cz+d=0  (3)

Next, operations of portions of the distance measurement device 10Aaccording to the technique of this disclosure will be described.

First, reference will be made to FIGS. 11 and 12 to describe an imagingposition distance calculation process realized by the CPU 100 executingthe imaging position distance calculation program 106 in a case wherethe three-dimensional coordinate calculation button 90G is turned on.

Meanwhile, hereinafter, for convenience of description, a descriptionwill be given on the assumption that a region including an outer wallsurface 121 of an office building 120 is included as a subject in animaging range 119 of the imaging device 14 of the distance measurementdevice 10A, as illustrated in FIG. 13 as an example. In addition, adescription will be given on the assumption that the outer wall surface121 is a main subject and is an object to be irradiated with a laserbeam.

In addition, the outer wall surface 121 is formed to have a planarshape, and is an example of a planar region according to the techniqueof this disclosure. In addition, as illustrated in FIG. 13 as anexample, a plurality of windows 122 having a quadrilateral shape areprovided on the outer wall surface 121. In addition, as illustrated inFIG. 13 as an example, a pattern 124 having a laterally long rectangularshape is drawn below each window 122 on the outer wall surface 121.However, the invention is not limited thereto, and dirt attached to theouter wall surface 121, a crack, or the like may be used.

Meanwhile, in this embodiment, the “planar shape” includes not only aplane but also a planar shape in a range allowing slight irregularitiesdue to the window, a ventilating opening, or the like, and may be, forexample, a plane or a planar shape which is recognized as a “planarshape” by visual observation or the existing image analysis technique.

In addition, hereinafter, for convenience of description, a descriptionwill be given on the assumption that a distance to the outer wallsurface 121 is measured by the distance measurement device 10A by alaser beam being emitted to the outer wall surface 121. In addition,hereinafter, for convenience of description, the position of thedistance measurement device 10A in a case where the distance measurementunit 12 is positioned at a first measurement position and the imagingdevice 14 is positioned at a first imaging position will be referred toas a “first position”. In addition, hereinafter, for convenience ofdescription, the position of the distance measurement device 10A in acase where the distance measurement unit 12 is positioned at a secondmeasurement position and the imaging device 14 is positioned at a secondimaging position will be referred to as a “second position”.

In the imaging position distance calculation process illustrated in FIG.11, first, the acquisition unit 110 determines whether or not themeasurement and imaging of a distance have been executed at the firstposition by the distance measurement device 10A, in step 200. The firstposition may be a position where the outer wall surface 121 can beirradiated with a laser beam and a region including the outer wallsurface 121 can be imaged as a subject.

In a case where the measurement and imaging of a distance have not beenexecuted at the first position by the distance measurement device 10A instep 200, the determination result is negative, and the process proceedsto step 201. In a case where the measurement and imaging of a distancehave been executed at the first position by the distance measurementdevice 10A in step 200, the determination result is positive, and theprocess proceeds to step 202.

In step 201, the acquisition unit 110 determines whether or not acondition for terminating the imaging position distance calculationprocess has been satisfied. The condition for terminating the imagingposition distance calculation process refers to, for example, acondition that an instruction for terminating the imaging positiondistance calculation process is received by the touch panel 88 or acondition that the determination result is not positive after the startof the processing of step 200 and a first predetermined time elapses.Meanwhile, the first predetermined time refers to, for example, oneminute.

In a case where the condition for terminating the imaging positiondistance calculation process has not been satisfied in step 201, thedetermination result is negative, and the process proceeds to step 200.In a case where the condition for terminating the imaging positiondistance calculation process has been satisfied in step 201, thedetermination result is positive, and thus the imaging position distancecalculation process is terminated.

In step 202, the acquisition unit 110 acquires a distance measured atthe first position and a first captured image signal indicating a firstcaptured image obtained by executing imaging at the first position, andthen the process proceeds to step 204. Meanwhile, the first capturedimage is a captured image obtained by performing imaging in a focusingstate at the first position.

In step 204, the acquisition unit 110 displays the acquired firstcaptured image indicated by the first captured image signal on thedisplay unit 86 as illustrated in FIG. 14 as an example, and then theprocess proceeds to step 206.

In step 206, the acquisition unit 110 determines whether or not anattention pixel has been designated by a user from the first capturedimage through the touch panel 88. Here, the attention pixel isequivalent to the above-described first designated pixel. Meanwhile, thetouch panel 88 receives pixel designation information (an example ofsecond pixel designation information according to the technique of thisdisclosure) for designating two-dimensional coordinates corresponding toa pixel included in the first captured image among two-dimensionalcoordinates given to the touch panel 88. Accordingly, it is determinedin step 206 that an attention pixel has been designated in a case wherethe pixel designation information is received by the touch panel 88.That is, a pixel corresponding to the two-dimensional coordinatesdesignated in accordance with the pixel designation information is anattention pixel.

In step 206, in a case where an attention pixel has not been designatedby the user from the first captured image through the touch panel 88,the determination result is negative, and the process proceeds to step208. In step 206, in a case where an attention pixel has been designatedby the user from the first captured image through the touch panel 88,the determination result is positive, and the process proceeds to step210.

In step 208, the acquisition unit 110 determines whether or not acondition for terminating the imaging position distance calculationprocess has been satisfied. In step 208, in a case where the conditionfor terminating the imaging position distance calculation process hasnot been satisfied, the determination result is negative, and theprocess proceeds to step 206. In step 208, in a case where the conditionfor terminating the imaging position distance calculation process hasbeen satisfied, the determination result is positive, and thus theimaging position distance calculation process is terminated.

In step 210, the acquisition unit 110 acquires attention pixelcoordinates for specifying an attention pixel 126 (see FIG. 14) which isdesignated by the user through the touch panel 88 in the first capturedimage, and then the process proceeds to step 212. Meanwhile, here, anexample of the pixel designated by the user through the touch panel 88is the attention pixel 126 as illustrated in FIG. 14 as an example. Theattention pixel 126 is a pixel at the lower left corner in a front viewof an image equivalent to a central window in a second floor of theouter wall surface in the first captured image, as illustrated in FIG.14 as an example. The central window in the second floor of the outerwall surface indicates a central window 122 in a second floor of theoffice building 120 among the windows 122 provided on the outer wallsurface 121 in the example illustrated in FIG. 13. In addition, theattention pixel coordinates indicate two-dimensional coordinates forspecifying the attention pixel 126 in the first captured image.

In step 212, the acquisition unit 110 acquires three characteristicpixel coordinates for specifying three characteristic pixels in an outerwall surface image 128 (a hatched region in the example illustrated inFIG. 15) in the first captured image, and then the process proceeds tostep 214. Meanwhile, the “three characteristic pixels” as mentionedherein is an example of “a plurality of pixels” and “a plurality ofcharacteristic pixels” according to the technique of this disclosure.

Here, the outer wall surface image 128 refers to an image showing theouter wall surface 121 (see FIG. 13) in the first captured image. Thethree characteristic pixels are pixels which are separated from eachother by a predetermined number of pixels within the first capturedimage and are respectively present at three points specified inaccordance with a fixed rule by image analysis on the basis of a spatialfrequency and the like of an image equivalent to a pattern, a buildingmaterial, or the like in the outer wall surface image 128. For example,three pixels which show different apexes having a maximum spatialfrequency within a circular region, which is fixed by a predeterminedradius on the basis of the attention pixel 126, and satisfy fixedconditions are extracted as three characteristic pixels. Meanwhile, thethree characteristic pixel coordinates are equivalent to theabove-described plurality of pixel coordinates.

In the example illustrated in FIG. 15, the three characteristic pixelsare a first pixel 130, a second pixel 132, and a third pixel 134. Thefirst pixel 130 is a pixel at the upper left corner in a front view ofan image equivalent to a central window in the second floor of the outerwall surface in the outer wall surface image 128. The second pixel 132is a pixel at the upper right corner in a front view of the imageequivalent to the central window in the second floor of the outer wallsurface. The third pixel 134 is a pixel at the lower left corner in afront view of an image equivalent to the pattern 124 close to a lowerportion of a central window in a third floor of the outer wall surface.Meanwhile, the central window in the third floor of the outer wallsurface refers to a central window 122 in the third floor of the officebuilding 120 among the windows 122 provided on the outer wall surface121, in the example illustrated in FIG. 13.

In step 214, the derivation unit 112 calculates emission positioncoordinates from the distance L, the half angle of view α, the emissionangle β, and the distance between reference points M on the basis ofExpression (2), and then the process proceeds to step 216. The distanceL used in the processing of step 214 refers to a distance to the subjectwhich is measured at the first imaging position by the distancemeasurement device 10A.

In step 216, the derivation unit 112 displays a distance and emissionposition mark 136 on the display unit 86 so as to be superimposed on thefirst captured image as illustrated in FIG. 16 as an example, and thenthe process proceeds to step 218.

The distance displayed by the execution of the processing of step 216indicates a distance which is measured at the first imaging position bythe distance measurement device 10A, that is, the distance L which isused for the calculation of the emission position coordinates in theprocessing of step 214. In the example illustrated in FIG. 16, anumerical value of “133325.0” corresponds to the distance L which ismeasured at the first imaging position by the distance measurementdevice 10A, and the unit is millimeter.

In the example illustrated in FIG. 16, the emission position mark 136 isa mark indicating a position which is specified by the emission positioncoordinates calculated by the execution of the processing of step 214.

In step 218 illustrated in FIG. 12, the acquisition unit 110 determineswhether or not imaging has been executed at the second position by thedistance measurement device 10A. The second position is a position of amoving destination of the distance measurement device 10A, and may be aposition where the outer wall surface 121 can be irradiated with a laserbeam and a region including the outer wall surface 121 can be imaged asa subject.

In step 218, in a case where imaging has not been executed at the secondposition by the distance measurement device 10A, the determinationresult is negative, and the process proceeds to step 220. In step 218,in a case where imaging has been executed at the second position by thedistance measurement device 10A, the determination result is positive,and the process proceeds to step 222.

In step 220, the acquisition unit 110 determines whether or not acondition for terminating the imaging position distance calculationprocess has been satisfied. In step 220, in a case where the conditionfor terminating the imaging position distance calculation process hasnot been satisfied, the determination result is negative, and theprocess proceeds to step 218. In step 220, in a case where the conditionfor terminating the imaging position distance calculation process hasbeen satisfied, the determination result is positive, and thus theimaging position distance calculation process is terminated.

In step 222, the acquisition unit 110 acquires a second captured imagesignal indicating the second captured image obtained by executingimaging at the second position, and then the process proceeds to step224. Meanwhile, the second captured image is a captured image obtainedby performing imaging in a focusing state at the second position.

In step 224, the acquisition unit 110 displays the acquired secondcaptured image indicated by the second captured image signal on thedisplay unit 86, and then the process proceeds to step 226.

In step 226, the acquisition unit 110 specifies a correspondingattention pixel which is a pixel corresponding to the attention pixel126 among pixels included in the second captured image and acquirescorresponding attention pixel coordinates for specifying the specifiedcorresponding attention pixel, and then the process proceeds to step228. Meanwhile, here, the corresponding attention pixel coordinatesrefer to two-dimensional coordinates for specifying the correspondingattention pixel in the second captured image. In addition, thecorresponding attention pixel is specified by executing the existingimage analysis by using the first and second captured images as objectsto be analyzed. Meanwhile, the corresponding attention pixel isequivalent to the above-described second designated pixel, and isuniquely specified from the second captured image by the execution ofthe processing of step 226 when the attention pixel 126 is specifiedfrom the first captured image.

In step 228, the acquisition unit 110 specifies three characteristicpixels in an outer wall surface image corresponding to the outer wallsurface image 128 (see FIG. 15) in the second captured image andacquires corresponding characteristic pixel coordinates for specifyingthe specified three characteristic pixels, and then the process proceedsto step 230. Meanwhile, the “three characteristic pixels” as mentionedherein is an example of “a plurality of pixels” and “a plurality ofcharacteristic pixels” according to the technique of this disclosure. Inaddition, the corresponding characteristic pixel coordinates refer totwo-dimensional coordinates for specifying the three characteristicpixels specified in the second captured image. In addition, thecorresponding characteristic pixel coordinates are also two-dimensionalcoordinates corresponding to the three characteristic pixel coordinatesacquired in the processing of step 212 in the second captured image, andare equivalent to the above-described plurality of pixel coordinates. Inaddition, the three characteristic pixels in the second captured imageare specified by executing the existing image analysis by using thefirst and second captured images as objects to be analyzed, similar tothe above-described method of specifying a corresponding attentionpixel.

In step 230, the derivation unit 112 derives a, b, and c of the planeequation shown in Expression (3) from the three characteristic pixelcoordinates, the corresponding characteristic pixel coordinates, thefocal length of the imaging lens 50, and the dimension of the imagingpixel 60A1 to derive the direction of a plane specified by the planeequation.

Here, the three characteristic pixel coordinates are set to be (u_(L1),v_(L1)), (u_(L2), v_(L2)), and (u_(L3), v_(L3)) and the correspondingcharacteristic pixel coordinates are set to be (u_(R1), v_(R1)),(u_(R2), v_(R2)), and (u_(R3), v_(R3)), first to third characteristicpixel three-dimensional coordinates are specified by the followingExpressions (4) to (6). The first characteristic pixel three-dimensionalcoordinates refer to three-dimensional coordinates corresponding to(u_(L1), v_(L1)) and (u_(R1), v_(R1)). The second characteristic pixelthree-dimensional coordinates refer to three-dimensional coordinatescorresponding to (u_(L2), v_(L2)) and (u_(R2), v_(R2)). The thirdcharacteristic pixel three-dimensional coordinates indicatethree-dimensional coordinates corresponding to (u_(L3), v_(L3)) and(u_(R3), v_(R3)). Meanwhile, in Expression (4) to (6), “v_(R1)”,“v_(R2)”, and “v_(R3)” are not used.

$\begin{matrix}{{{first}\mspace{14mu}{characteristic}\mspace{14mu}{pixel}\mspace{14mu}{three}\text{-}{dimensional}\mspace{14mu}{coordinates}\text{:}}\left( {{\frac{B}{u_{L\; 1} - u_{R\; 1}}u_{L\; 1}},{\frac{B}{u_{L\; 1} - u_{R\; 1}}v_{L\; 1}},{\frac{B}{u_{L\; 1} - u_{R\; 1}}f}} \right)} & (4) \\{{{second}\mspace{14mu}{characteristic}\mspace{14mu}{pixel}\mspace{14mu}{three}\text{-}{dimensional}\mspace{14mu}{coordinates}\text{:}}\left( {{\frac{B}{u_{L\; 2} - u_{R\; 2}}u_{L\; 2}},{\frac{B}{u_{L\; 2} - u_{R\; 2}}v_{L\; 2}},{\frac{B}{u_{L\; 2} - u_{R\; 2}}f}} \right)} & (5) \\{{{third}\mspace{14mu}{characteristic}\mspace{14mu}{pixel}\mspace{14mu}{three}\text{-}{dimensional}\mspace{14mu}{coordinates}\text{:}}\left( {{\frac{B}{u_{L\; 3} - u_{R\; 3}}u_{L\; 3}},{\frac{B}{u_{L\; 3} - u_{R\; 3}}v_{L\; 3}},{\frac{B}{u_{L\; 3} - u_{R\; 3}}f}} \right)} & (6)\end{matrix}$

In step 230, the derivation unit 112 derives a, b, and c in Expression(3) by optimizing a, b, and c in Expression (3) from three expressionshaving an equivalence relationship obtained by substituting each of thefirst to third characteristic pixel three-dimensional coordinates shownin Expressions (4) to (6) for Expression (3). In this manner, thederivation of a, b, and c in Expression (3) means that the direction ofthe plane specified by the plane equation shown in Expression (3) isderived.

In step 232, the derivation unit 112 decides the plane equation shown inExpression (3) on the basis of the emission position coordinates derivedin the processing of step 214, and then the process proceeds to step234. That is, in step 232, the derivation unit 112 substitutes a, b, andc derived in the processing of step 230 and the emission positioncoordinates derived in the processing of step 214 for Expression (3) todecide d in Expression (3). Since a, b, and c in Expression (3) arederived in the processing of step 230, the plane equation shown inExpression (3) is decided when d in Expression (3) is decided in theprocessing of step 232.

In step 234, the derivation unit 112 calculates an imaging positiondistance on the basis of the attention pixel coordinates, thecorresponding attention pixel coordinates, the focal length of theimaging lens 50, the dimension of the imaging pixel 60A1, the planeequation, and Expression (1), and then the process proceeds to step 236.

Here, the attention pixel coordinates used in the processing of step 234refer to the attention pixel coordinates acquired in the processing ofstep 210. In addition, the corresponding attention pixel coordinatesused in the processing of step 234 refer to the corresponding attentionpixel coordinates acquired in the processing of step 226. Further, theplane equation used in step 234 refers to the plane equation decided instep 232.

Accordingly, in step 234, (X, Y, Z) in Expression (1) for which theattention pixel coordinates, the corresponding attention pixelcoordinates, the focal length of the imaging lens 50, and the dimensionof the imaging pixel 60A1 are substituted is substituted for the planeequation, and thus “B” which is an imaging position distance is derived.

Meanwhile, “B” which is an imaging position distance may be derived onthe basis of the characteristic pixel three-dimensional coordinates andthe plane equation decided in step 232. That is, in this case, “B” whichis an imaging position distance is derived by the characteristic pixelthree-dimensional coordinates being substituted for the plane equationdecided in step 232. The “characteristic pixel three-dimensionalcoordinates” as mentioned herein refers to, for example, the firstcharacteristic pixel three-dimensional coordinates. However, theinvention is not limited thereto, and the characteristic pixelthree-dimensional coordinates may be the second characteristic pixelthree-dimensional coordinates or the third characteristic pixelthree-dimensional coordinates.

In step 236, the control unit 114 displays the imaging position distancecalculated in the processing of step 234 on the display unit 86 so as tobe superimposed on the second captured image, as illustrated in FIG. 17as an example. In step 236, the control unit 114 stores the imagingposition distance calculated in the processing of step 234 in apredetermined storage region, and then terminates the imaging positiondistance calculation process. Meanwhile, an example of the predeterminedstorage region is a storage region of the primary storage unit 102 or astorage region of the secondary storage unit 104.

Meanwhile, in the example illustrated in FIG. 17, a numerical value of“144656.1” corresponds to the imaging position distance calculated inthe processing of step 234, and the unit is millimeter.

Next, reference will be made to FIG. 18 to describe thethree-dimensional coordinate calculation process realized by the CPU 100executing the three-dimensional coordinate calculation program 108 in acase where the three-dimensional coordinate calculation button 90G isturned on.

In the three-dimensional coordinate calculation process illustrated inFIG. 18, first, in step 250, the derivation unit 112 determines whetheror not an imaging position distance has been already calculated in theprocessing of step 234 included in the imaging position distancecalculation process. In step 250, in a case where an imaging positiondistance has not been calculated in the processing of step 234 includedin the imaging position distance calculation process, the determinationresult is negative, and the process proceeds to step 258. In step 250,in a case where an imaging position distance has been already calculatedin the processing of step 234 included in the imaging position distancecalculation process, the determination result is positive, and theprocess proceeds to step 252.

In step 252, the derivation unit 112 determines whether or not acondition (hereinafter, referred to as a “calculation start condition”)for starting the calculation of designated pixel three-dimensionalcoordinates has been satisfied. An example of the calculation startcondition is a condition that an instruction for starting thecalculation of the designated pixel three-dimensional coordinates isreceived by the touch panel 88, or a condition that the imaging positiondistance is displayed on the display unit 86.

In step 252, in a case where the calculation start condition has notbeen satisfied, the determination result is negative, and the processproceeds to step 258. In step 252, in a case where the calculation startcondition has been satisfied, the determination result is positive, andthe process proceeds to step 254.

In step 254, the derivation unit 112 calculates designated pixelthree-dimensional coordinates on the basis of the attention pixelcoordinates, the corresponding attention pixel coordinates, the imagingposition distance, the focal length of the imaging lens 50, thedimension of the imaging pixel 60A1, and Expression (1), and then theprocess proceeds to step 256.

Here, the attention pixel coordinates used in the processing of step 254refer to the attention pixel coordinates acquired in the processing ofstep 210 included in the imaging position distance calculation process.In addition, the corresponding attention pixel coordinates used in theprocessing of step 254 refer to the corresponding attention pixelcoordinates acquired in the processing of step 226 included in theimaging position distance calculation process. In addition, the imagingposition distance used in the processing of step 254 indicates theimaging position distance derived in the processing of step 234 includedin the imaging position distance calculation process.

Accordingly, in step 254, the designated pixel three-dimensionalcoordinates are calculated by substituting the attention pixelcoordinates, the corresponding attention pixel coordinates, the imagingposition distance, the focal length of the imaging lens 50, and thedimension of the imaging pixel 60A1 for Expression (1).

In step 256, the control unit 114 displays the designated pixelthree-dimensional coordinates calculated in the processing of step 254on the display unit 86 so as to be superimposed on the second capturedimage, as illustrated in FIG. 19 as an example. In step 256, the controlunit 114 stores the designated pixel three-dimensional coordinatescalculated in the processing of step 254 in a predetermined storageregion, and then terminates the three-dimensional coordinate calculationprocess. Meanwhile, an example of the predetermined storage region is astorage region of the primary storage unit 102 and a storage region ofthe secondary storage unit 104.

Meanwhile, in the example illustrated in FIG. 19, (20161, 50134, 136892)corresponds to the designated pixel three-dimensional coordinatescalculated in the processing of step 254. In the example illustrated inFIG. 19, the designated pixel three-dimensional coordinates aredisplayed in proximity to the attention pixel 126. Meanwhile, theattention pixel 126 may be emphatically displayed so as to bedistinguishable from other pixels.

In step 258, the derivation unit 112 determines whether or not acondition for terminating the three-dimensional coordinate calculationprocess has been satisfied. An example of the condition for terminatingthe three-dimensional coordinate calculation process is a condition thatan instruction for terminating the three-dimensional coordinatecalculation process is received by the touch panel 88. Another exampleof the condition for terminating the three-dimensional coordinatecalculation process is a condition that the determination result is notpositive in step 250 after the determination result is negative in step250 and a second predetermined time elapses, and the like. Meanwhile,the second predetermined time refers to, for example, 30 minutes.

In step 258, in a case where the condition for terminating thethree-dimensional coordinate calculation process has not been satisfied,the determination result is negative, and the process proceeds to step250. In step 258, in a case where the condition for terminating thethree-dimensional coordinate calculation process has been satisfied, thedetermination result is positive, and thus the three-dimensionalcoordinate calculation process is terminated.

As described above, in the distance measurement device 10A, the firstcaptured image, the second captured image, and the distance to thesubject are acquired by the acquisition unit 110. In addition, theattention pixel 126 is designated in the first captured image by theuser through the touch panel 88, and attention pixel coordinates areacquired by the acquisition unit 110 (step 210). In addition,corresponding attention pixel coordinates are acquired by theacquisition unit 110 (step 226). In addition, three characteristic pixelcoordinates are acquired by the acquisition unit 110 (step 212). Inaddition, corresponding characteristic pixel coordinates are acquired bythe acquisition unit 110 (step 228). In addition, the emission positioncoordinates are calculated by the derivation unit 112 (step 214). Theimaging position distance is derived by the derivation unit 112 on thebasis of the attention pixel coordinates, the corresponding attentionpixel coordinates, the three characteristic pixel coordinates,corresponding characteristic pixel coordinates, the emission positioncoordinates, the focal length of the imaging lens 50, and the dimensionof the imaging pixel 60A1.

Therefore, according to the distance measurement device 10A, even when acharacteristic location capable of being specified is not irradiatedwith a laser beam, it is possible to derive the imaging positiondistance on the basis of the first captured image and the secondcaptured image which are obtained by respectively imaging the subjectfrom the first imaging position and the second imaging position.

In the distance measurement device 10A, designated pixelthree-dimensional coordinates are calculated on the basis of the imagingposition distance calculated in the imaging position distancecalculation process (see FIG. 18). Therefore, according to the distancemeasurement device 10A, even when a characteristic location capable ofbeing specified is not irradiated with a laser beam, the designatedpixel three-dimensional coordinates can be derived.

In the distance measurement device 10A, the designated pixelthree-dimensional coordinates are specified on the basis of theattention pixel coordinates, the corresponding attention pixelcoordinates, the imaging position distance, the focal length of theimaging lens 50, and the dimension of the imaging pixel 60A1 (seeExpression (1)). Therefore, according to the distance measurement device10A, it is possible to derive the designated pixel three-dimensionalcoordinates with a high level of accuracy, as compared to a case wherethe designated pixel three-dimensional coordinates are not specified onthe basis of the attention pixel coordinates, the correspondingattention pixel coordinates, the imaging position distance, the focallength of the imaging lens 50, and the dimension of the imaging pixel60A1.

In the distance measurement device 10A, the direction of a planespecified by the plane equation shown in Expression (3) is derived bythe derivation unit 112 on the basis of the three characteristic pixelcoordinates, the corresponding characteristic pixel coordinates, thefocal length of the imaging lens 50, and the dimension of the imagingpixel 60A1 (step 230). In addition, the plane equation shown inExpression (3) is decided by the derivation unit 112 on the basis of thedirection of the plane and the emission position coordinates calculatedin the processing of step 214 (step 232). An imaging position distanceis calculated by the derivation unit 112 on the basis of the decidedplane equation, the attention pixel coordinates, the focal length of theimaging lens 50, and the dimension of the imaging pixel 60A1 (step 234).Therefore, according to the distance measurement device 10A, it ispossible to derive the imaging position distance with a high level ofaccuracy, as compared to a case where the imaging position distance isderived without using the plane equation when a characteristic locationcapable of being specified is not irradiated with a laser beam.

In the distance measurement device 10A, three characteristic pixelcoordinates are acquired by the acquisition unit 110 (step 212), andcorresponding characteristic pixel coordinates are acquired by theacquisition unit 110 (step 228). An imaging position distance iscalculated by the derivation unit 112 on the basis of the attentionpixel coordinates, the corresponding attention pixel coordinates, thethree characteristic pixel coordinates, the corresponding characteristicpixel coordinates, the emission position coordinates, the focal lengthof the imaging lens 50, and the dimension of the imaging pixel 60A1(steps 230 to 234). Therefore, according to the distance measurementdevice 10A, it is possible to derive the imaging position distance onthe basis of the three characteristic pixel coordinates and thecorresponding characteristic pixel coordinates with a small number ofoperations, as compared to a case where the user designates threecharacteristic pixels in acquiring the three characteristic pixelcoordinates and the corresponding characteristic pixel coordinates.

In the distance measurement device 10A, pixel designation information isreceived by the touch panel 88, a pixel designated on the basis of thereceived pixel designation information is set to be the attention pixel126, and attention pixel coordinates are acquired by the acquisitionunit 110 (step 210). In addition, a corresponding attention pixel whichis a pixel corresponding to the attention pixel 126 is specified by theacquisition unit 110. Corresponding attention pixel coordinates forspecifying the corresponding attention pixel are acquired by theacquisition unit 110 (step 226). Therefore, according to the distancemeasurement device 10A, it is possible to rapidly determine a designatedpixel related to both the first captured image and the second capturedimage, as compared to a case where the designated pixel related to boththe first captured image and the second captured image is designated bythe user.

In addition, the distance measurement device 10A includes the distancemeasurement unit 12 and the distance measurement control unit 68, and adistance to the subject which is measured by the distance measurementunit 12 and the distance measurement control unit 68 is acquired by theacquisition unit 110. Therefore, according to the distance measurementdevice 10A, it is possible to easily acquire a distance to the subjectwhich is used for the derivation of emission position coordinates, ascompared to a case where the distance measurement device does notinclude the distance measurement unit 12 and the distance measurementcontrol unit 68.

In addition, the distance measurement device 10A includes the imagingdevice 14, and the first captured image and the second captured imagewhich are obtained by imaging the subject by the imaging device 14 areacquired by the acquisition unit 110. Therefore, according to thedistance measurement device 10A, it is possible to easily acquire thefirst captured image and the second captured image which are used toobtain the attention pixel coordinates, the three characteristic pixelcoordinates, the corresponding attention pixel coordinates, and thecorresponding characteristic pixel coordinates, as compared to a casewhere the distance measurement device does not include the imagingdevice 14.

Further, in the distance measurement device 10A, derivation results ofthe derivation unit 112 are displayed by the display unit 86. Therefore,according to the distance measurement device 10A, it is possible to makethe user easily recognize the derivation results of the derivation unit112, as compared to a case where the derivation results of thederivation unit 112 are not displayed by the display unit 86.

Meanwhile, in the first embodiment, the three characteristic pixelcoordinates are described, but the technique of this disclosure is notlimited thereto. For example, two-dimensional coordinates for specifyingeach of a predetermined number of pixels more than four characteristicpixels may be adopted instead of the three characteristic pixelcoordinates.

In the first embodiment, a description has been given of a case wherethe attention pixel coordinates are acquired from coordinates on thefirst captured image and the corresponding attention pixel coordinatesare acquired from coordinates on the second captured image, but thetechnique of this disclosure is not limited thereto. For example, theattention pixel coordinates may be acquired from the coordinates on thesecond captured image, and the corresponding attention pixel coordinatesmay be acquired from the coordinates on the first captured image.

In the first embodiment, a description has been given of a case wherethe three characteristic pixel coordinates are acquired from coordinateson the first captured image and the corresponding characteristic pixelcoordinates are acquired from coordinates on the second captured image,but the technique of this disclosure is not limited thereto. Forexample, the three characteristic pixel coordinates may be acquired fromthe coordinates on the second captured image, and the correspondingcharacteristic pixel coordinates may be acquired from the coordinates onthe first captured image.

In the first embodiment, a description has been given of a case wheretwo-dimensional coordinates for specifying each of the first pixel 130,the second pixel 132, and the third pixel 134 are acquired by theacquisition unit 110 as three characteristic pixel coordinates, but thetechnique of this disclosure is not limited thereto. For example, asillustrated in FIG. 20, two-dimensional coordinates for specifying eachof a first pixel 130A, a second pixel 132A, and a third pixel 134A maybe acquired by the acquisition unit 110. The first pixel 130A, thesecond pixel 132A, and the third pixel 134A are three pixels formaximizing an area surrounded in the outer wall surface image 128.Meanwhile, the invention is not limited to the three pixels, and thepixels may be a predetermined number of pixels more than three pixelsfor maximizing an area surrounded in the outer wall surface image 128.

In this manner, in the example illustrated in FIG. 20, three pixels formaximizing an area surrounded in the outer wall surface image 128 arespecified as three characteristic pixels, and two-dimensionalcoordinates related to the specified the three pixels are acquired bythe acquisition unit 110 as three characteristic pixel coordinates. Inaddition, corresponding characteristic pixel coordinates correspondingto the three characteristic pixel coordinates are also acquired by theacquisition unit 110. Therefore, according to the distance measurementdevice 10A, it is possible to derive an imaging position distance with ahigh level of accuracy, as compared to a case where three characteristicpixel coordinates for specifying a plurality of pixels not formaximizing an area surrounded and corresponding characteristic pixelcoordinates are acquired as three characteristic pixels.

In the first embodiment, a description has been given of a case wherethe imaging position distance derivation process is realized when thethree-dimensional coordinate calculation button 90G is turned on, butthe invention is not limited thereto. For example, the imaging positiondistance derivation process may be executed in a case where the imagingposition distance calculation button 90F is turned on. The imagingposition distance derivation process described in the first embodimentis an example in a case where the derivation of three-dimensionalcoordinates is set to be the final purpose.

For this reason, attention pixel coordinates and corresponding pixelcoordinates which are required in the derivation of three-dimensionalcoordinates are acquired through the imaging position distancederivation process. However, in a case where only the derivation of animaging position distance is a purpose, it is not necessary to acquireattention pixel coordinates and corresponding pixel coordinates in theimaging position distance derivation process. Accordingly, the CPU 100may derive the imaging position distance without acquiring the attentionpixel coordinates and the corresponding attention pixel coordinates in acase where the imaging position distance calculation button 90F isturned on, and may then acquire the attention pixel coordinates and thecorresponding attention pixel coordinates in a case where thethree-dimensional coordinate calculation button 90G is turned on. Inthis case, the CPU 100 may acquire the attention pixel coordinates andthe corresponding attention pixel coordinates, for example, between theprocessing of step 252 and the processing of step 254 of thethree-dimensional coordinates derivation process illustrated in FIG. 34,and may use the acquired attention pixel coordinates and correspondingattention pixel coordinates in the processing of step 254.

Second Embodiment

In the first embodiment, a description has been given of a case wherethree characteristic pixel coordinates are acquired with respect to theentire outer wall surface image 128. However, in a second embodiment, adescription will be given of a case where three characteristic pixelcoordinates are acquired with respect to a portion of the outer wallsurface image 128. Meanwhile, in the second embodiment, the samecomponents as those described in the first embodiment will be denoted bythe same reference numerals and signs, and a description thereof will beomitted.

A distance measurement device 10B according to the second embodiment isdifferent from the distance measurement device 10A as illustrated inFIG. 6 as an example in that an imaging position distance calculationprogram 150 is stored in a secondary storage unit 104 instead of theimaging position distance calculation program 106.

The CPU 100 executes the imaging position distance calculation program150 and a three-dimensional coordinate calculation program 108 to beoperated as an acquisition unit 154, a derivation unit 112, and acontrol unit 156 which is an example of each of a first control unit anda second control unit according to the technique of this disclosure (seeFIG. 9).

The acquisition unit 154 corresponds to the acquisition unit 110described in the first embodiment, and the control unit 156 correspondsto the control unit 114 described in the first embodiment. Meanwhile, inthe second embodiment, for convenience of description, differentportions from the acquisition unit 110 and the control unit 114described in the first embodiment will be described with regard to theacquisition unit 154 and the control unit 156.

The control unit 156 performs control of displaying a first capturedimage on a display unit 86 and displaying an outer wall surface image128, which is an example of a corresponding region according to thetechnique of this disclosure, within a display region so as to bedistinguishable from the other regions. A touch panel 88 receives regiondesignation information for designating a coordinate acquisition targetregion 158 (see FIG. 22) in a state where the outer wall surface image128 is displayed on the display unit 86. Here, the coordinateacquisition target region 158 refers to a partial closed region in theouter wall surface image 128. The region designation information refersto information for designating the coordinate acquisition target region158.

The acquisition unit 154 acquires three characteristic pixel coordinatesfrom the coordinate acquisition target region 158 designated inaccordance with the region designation information received by the touchpanel 88.

Next, an imaging position distance calculation process realized by theCPU 100 executing the imaging position distance calculation program 150will be described with reference to FIG. 21, as the operation ofportions of the distance measurement device 10B according to thetechnique of this disclosure. Meanwhile, the same steps as those in theflowchart illustrated in FIG. 11 will be denoted by the same stepnumbers, and a description thereof will be omitted.

The flowchart illustrated in FIG. 21 is different from the flowchartillustrated in FIG. 11 in that steps 300 to 312 are provided instead ofstep 212.

In step 300 illustrated in FIG. 21, the control unit 156 specifies theouter wall surface image 128 (see FIG. 15) from the first capturedimage, and then the process proceeds to step 302.

In step 302, the control unit 156 emphatically displays the outer wallsurface image 128 specified in the processing of step 300 on the displayunit 86 so as to be distinguishable from the other regions within thedisplay region of the first captured image, and then the processproceeds to step 304.

In step 304, the acquisition unit 154 determines whether or not theregion designation information has been received by the touch panel 88and the coordinate acquisition target region 158 has been designated inaccordance with the received region designation information.

In step 304, in a case where the coordinate acquisition target region158 has not been designated in accordance with the region designationinformation, the determination result is negative, and the processproceeds to step 306. In step 304, in a case where the coordinateacquisition target region 158 has been designated in accordance with theregion designation information, the determination result is positive,and the process proceeds to step 308.

In step 306, the acquisition unit 154 determines whether or not acondition for terminating the imaging position distance calculationprocess has been satisfied. In step 306, in a case where the conditionfor terminating the imaging position distance calculation process hasnot been satisfied, the determination result is negative, and theprocess proceeds to step 304. In step 306, in a case where the conditionfor terminating the imaging position distance calculation process hasbeen satisfied, the determination result is positive, and thus theimaging position distance calculation process is terminated.

In step 308, the acquisition unit 154 determines whether or not thecoordinate acquisition target region 158 designated in accordance withthe region designation information received by the touch panel 88includes the three characteristic pixels described in the firstembodiment.

As illustrated in FIG. 22 as an example, in a case where the coordinateacquisition target region 158 has been designated in accordance with theregion designation information received by the touch panel 88, thecoordinate acquisition target region 158 includes a pattern image 160showing a pattern 124 (see FIG. 13).

In the example illustrated in FIG. 23, the coordinate acquisition targetregion 158 includes a first pixel 162, a second pixel 164, and a thirdpixel 166 as three characteristic pixels. In the example illustrated inFIG. 23, the first pixel 162 is a pixel at the upper left corner in afront view of the pattern image 160, the second pixel 164 is a pixel atthe lower left corner in a front view of the pattern image 160, and thethird pixel 166 is a pixel at the lower right corner in a front view ofthe pattern image 160.

In step 308, in a case where the coordinate acquisition target region158 has been designated in accordance with the region designationinformation received by the touch panel 88 does not include threecharacteristic pixels, the determination result is negative, and theprocess proceeds to step 310. In step 308, in a case where thecoordinate acquisition target region 158 has been designated inaccordance with the region designation information received by the touchpanel 88 includes three characteristic pixels, the determination resultis positive, and the process proceeds to step 312. Meanwhile, the casewhere the determination result is positive in step 308 refers to a casewhere a region including the pattern image 160 has been designated inaccordance with the region designation information received by the touchpanel 88, for example, as illustrated in FIG. 22.

In step 310, the control unit 156 displays a re-designation message onthe display unit 86 so as to be superimposed on a predetermined regionof the first captured image, and then the process proceeds to step 304.The re-designation message refers to, for example, a message of “pleasedesignate a closed region including a characteristic pattern, a buildingmaterial, and the like”. Meanwhile, here, a case where there-designation message is visibly displayed has been described. However,the technique of this disclosure is not limited thereto, and audibledisplay such as the output of a sound using a sound reproducing device(not shown) or permanent visible display such as the output of printedmatter using a printer may be performed instead of the visible displayor may be performed in combination.

In step 312, the acquisition unit 154 acquires three characteristicpixel coordinates for specifying three characteristic pixels in thecoordinate acquisition target region designated in accordance with theregion designation information received by the touch panel 88, and thenthe process proceeds to step 214. Meanwhile, in the example illustratedin FIG. 23, the processing of step 312 is executed, and thustwo-dimensional coordinates for specifying each of the first pixel 162,the second pixel 164, and the third pixel 166 are acquired by theacquisition unit 154 as three characteristic pixel coordinates.

As described above, in the distance measurement device 10B, the outerwall surface image 128 is displayed on the display unit 86 so as to bedistinguishable from the other regions in the first captured image. Inaddition, the region designation information is received by the touchpanel 88, and a coordinate acquisition target region which is a portionof the outer wall surface image 128 is designated in accordance with thereceived region designation information. In a case where the coordinateacquisition target region includes three characteristic pixels, thethree characteristic pixel coordinates for specifying the threecharacteristic pixels are acquired by the acquisition unit 154 (step312), and corresponding characteristic pixel coordinates correspondingto the three characteristic pixel coordinates are also acquired (step228). Therefore, according to the distance measurement device 10B, it ispossible to acquire the three characteristic pixel coordinates and thecorresponding characteristic pixel coordinates with a small load, ascompared to a case where the three characteristic pixel coordinates andthe corresponding characteristic pixel coordinates are acquired withrespect to the entire outer wall surface image 128.

Third Embodiment

In the above-described embodiments, a description has been given of acase where three characteristic pixels are searched for and specifiedwithin a specific image through-image analysis. However, in a thirdembodiment, a description will be given of a case where threecharacteristic pixels are designated in accordance with an operation tothe touch panel 88. Meanwhile, in the third embodiment, the samecomponents as those described in the above-described embodiments will bedenoted by the same reference numerals and signs, and a descriptionthereof will be omitted.

A distance measurement device 10C according to the third embodiment isdifferent from the distance measurement device 10A in that an imagingposition distance calculation program 168 is stored in a secondarystorage unit 104 instead of the imaging position distance calculationprogram 106.

A CPU 100 executes the imaging position distance calculation program 168and a three-dimensional coordinate calculation program 108 to beoperated as an acquisition unit 172, a derivation unit 174, and acontrol unit 176 as illustrated in FIG. 9 as an example.

The acquisition unit 172 corresponds to the acquisition unit 110 (154)described in the above-described embodiments, the derivation unit 174corresponds to the derivation unit 112 described in the firstembodiment, and the control unit 176 corresponds to the control unit 114(156) described in the above-described embodiments. Meanwhile, in thethird embodiment, for convenience of description, different portionsfrom the acquisition unit 110 (154), the derivation unit 112, and thecontrol unit 114 (156) described in the above-described embodiments willbe described with regard to the acquisition unit 172, the derivationunit 174, and the control unit 176.

The touch panel 88 receives the pixel designation information (firstpixel designation information according to the technique of thisdisclosure) which is described in the first embodiment in a case whereeach of a first captured image and a second captured image is displayedon a display unit 86. In addition, the touch panel 88 also receives thepixel designation information (first pixel designation informationaccording to the technique of this disclosure) which is described in thefirst embodiment even when the second captured image is displayed on thedisplay unit 86.

In a case where the first captured image is displayed on the displayunit 86, the acquisition unit 110 acquires first characteristic pixelcoordinates which are two-dimensional coordinates for specifying each ofthree characteristic pixels designated in accordance with the pixeldesignation information received by the touch panel 88. The firstcharacteristic pixel coordinates are two-dimensional coordinatescorresponding to the three characteristic pixel coordinates described inthe first embodiment.

In a case where the second captured image is displayed on the displayunit 86, the acquisition unit 110 acquires second characteristic pixelcoordinates which are two-dimensional coordinates for specifying each ofthree characteristic pixels designated in accordance with the pixeldesignation information received by the touch panel 88. The secondcharacteristic pixel coordinates are two-dimensional coordinatescorresponding to the corresponding characteristic pixel coordinatesdescribed in the first embodiment.

The derivation unit 174 derives an imaging position distance on thebasis of attention pixel coordinates, corresponding attention pixelcoordinates, first characteristic pixel coordinates, secondcharacteristic pixel coordinates, emission position coordinates, a focallength of an imaging lens 50, and a dimension of an imaging pixel 60A1.

Next, an imaging position distance calculation process realized by theCPU 100 executing an imaging position distance calculation program 150will be described with reference to FIGS. 24 and 25, as the operation ofportions of the distance measurement device 10C according to thetechnique of this disclosure. Meanwhile, the same steps as those in theflowchart illustrated in FIGS. 12 and 21 will be denoted by the samestep numbers, and a description thereof will be omitted.

The flowchart illustrated in FIG. 24 is different from the flowchartillustrated in FIG. 21 in that step 349 is provided instead of step 304.In addition, the flowchart illustrated in FIG. 24 is different from theflowchart illustrated in FIG. 21 in that steps 350 and 352 are providedinstead of step 308. In addition, the flowchart illustrated in FIG. 24is different from the flowchart illustrated in FIG. 21 in that step 353is provided instead of step 310. In addition, the flowchart illustratedin FIG. 24 is different from the flowchart illustrated in FIG. 21 inthat step 354 is provided instead of step 312. Further, the flowchartillustrated in FIG. 25 is different from the flowchart illustrated inFIG. 12 in that steps 356 to 372 are provided instead of steps 228 and230.

In step 349, the acquisition unit 154 determines whether or not theregion designation information has been received by the touch panel 88and a first coordinate acquisition target region 178 (see FIG. 22) hasbeen designated in accordance with the received region designationinformation. Meanwhile, the first coordinate acquisition target region178 is a region corresponding to the coordinate acquisition targetregion described in the second embodiment.

In step 349, in a case where the first coordinate acquisition targetregion 178 has not been designated in accordance with the regiondesignation information, the determination result is negative, and theprocess proceeds to step 306. In step 349, in a case where the firstcoordinate acquisition target region 178 has been designated inaccordance with the region designation information, the determinationresult is positive, and the process proceeds to step 350.

In step 350, the control unit 176 emphatically displays the firstcoordinate acquisition target region 178, which is designated inaccordance with the region designation information received by the touchpanel, 88 on the display unit 86 so as to be distinguishable from theother regions within the display region of the first captured image, andthen the process proceeds to step 352.

In step 352, the acquisition unit 172 determines whether or not threecharacteristic pixels have been designated in accordance with the pixeldesignation information received by the touch panel 88.

As illustrated in FIG. 22 as an example, in a case where the firstcoordinate acquisition target region 178 has been designated inaccordance with the region designation information received by the touchpanel 88, the first coordinate acquisition target region 178 includes apattern image 160. In this case, the three characteristic pixels referto a first pixel 162, a second pixel 164, and a third pixel 166 whichare pixels positioned at three corners of the pattern image 160, asillustrated in FIG. 23 as an example.

In step 352, in a case where the three characteristic pixels have notbeen designated in accordance with the pixel designation informationreceived by the touch panel 88, the determination result is negative,and the process proceeds to step 353. In step 352, in a case where thethree characteristic pixels have been designated in accordance with thepixel designation information received by the touch panel 88, thedetermination result is positive, and the process proceeds to step 354.

In step 353, the control unit 176 displays a re-designation message onthe display unit 86 so as to be superimposed on a predetermined regionof the first captured image, and then the process proceeds to step 349.The re-designation message according to the third embodiment refers to,for example, a message of “please designate a closed region including acharacteristic pattern, a building material, and the like and thendesignate three characteristic pixels”.

In step 354, the acquisition unit 172 acquires first characteristicpixel coordinates for specifying the three characteristic pixelsdesignated in accordance with the pixel designation information receivedby the touch panel 88, and then the process proceeds to step 214.Meanwhile, in the example illustrated in FIG. 23, the processing of step354 is executed, and thus two-dimensional coordinates for specifyingeach of the first pixel 162, the second pixel 164, and the third pixel166 are acquired by the acquisition unit 172 as the first characteristicpixel coordinates.

In step 356 illustrated in FIG. 25, the control unit 176 specifies acorresponding outer wall surface image which is an outer wall surfaceimage corresponding to the outer wall surface image 128 from the secondcaptured image, and then the process proceeds to step 358.

In step 358, the control unit 176 emphatically displays thecorresponding outer wall surface image specified in the processing ofstep 356 on the display unit 86 so as to be distinguishable from theother regions within a display region of the second captured image, andthen the process proceeds to step 360.

In step 360, the acquisition unit 172 determines whether or not theregion designation information has been received by the touch panel 88and a second coordinate acquisition target region has been designated inaccordance with the received region designation information. Meanwhile,the second coordinate acquisition target region is a region designatedby the user through the touch panel 88 as a region corresponding to thefirst coordinate acquisition target region 178 (see FIG. 23) in thesecond captured image.

In step 360, in a case where the second coordinate acquisition targetregion has not been designated in accordance with the region designationinformation, the determination result is negative, and the processproceeds to step 362. In step 360, in a case where the second coordinateacquisition target region has been designated in accordance with theregion designation information, the determination result is positive,and the process proceeds to step 364.

In step 362, the acquisition unit 172 determines whether or not acondition for terminating the imaging position distance calculationprocess has been satisfied. In step 362, in a case where the conditionfor terminating the imaging position distance calculation process hasnot been satisfied, the determination result is negative, and theprocess proceeds to step 360. In step 362, in a case where the conditionfor terminating the imaging position distance calculation process hasbeen satisfied, the determination result is positive, and thus theimaging position distance calculation process is terminated.

In step 364, the control unit 176 emphatically displays the secondcoordinate acquisition target region, which is designated in accordancewith the region designation information received by the touch panel 88,on the display unit 86 so as to be distinguishable from the otherregions within the display region of the second captured image, and thenthe process proceeds to step 366.

In step 366, the acquisition unit 172 determines whether or not threecharacteristic pixels have been designated in accordance with the pixeldesignation information received by the touch panel 88.

In a case where the second coordinate acquisition target region has beendesignated in accordance with the region designation informationreceived by the touch panel 88, the second coordinate acquisition targetregion includes a pattern image corresponding to the pattern image 160.In this case, the three characteristic pixels are pixels positioned atthree corners of the pattern image corresponding to the pattern image160 in the second captured image. The pixels positioned at the threecorners of the pattern image corresponding to the pattern image 160refer to, for example, a pixel corresponding to the first pixel 162, apixel corresponding to the second pixel 164, and a pixel correspondingto the third pixel in the second captured image.

In step 366, in a case where the three characteristic pixels have notbeen designated in accordance with the pixel designation informationreceived by the touch panel 88, the determination result is negative,and the process proceeds to step 368. In step 366, in a case where thethree characteristic pixels have been designated in accordance with thepixel designation information received by the touch panel 88, thedetermination result is positive, and the process proceeds to step 370.

In step 368, the control unit 176 displays the re-designation messageaccording to the third embodiment on the display unit 86 so as to besuperimposed on a predetermined region of the second captured image, andthen the process proceeds to step 360.

In step 370, the acquisition unit 172 acquires second characteristicpixel coordinates for specifying the three characteristic pixelsdesignated in accordance with the pixel designation information receivedby the touch panel 88, and then the process proceeds to step 372.Meanwhile, in step 370, two-dimensional coordinates for specifying eachof the pixel corresponding to the first pixel 162, the pixelcorresponding to the second pixel 164, and the pixel corresponding tothe third pixel 166 are acquired by the acquisition unit 172 as thesecond characteristic pixel coordinates, for example, in the secondcaptured image.

In step 372, the derivation unit 174 derives a, b, and c of the planeequation shown in Expression (3) from the first characteristic pixelcoordinates, the second characteristic pixel coordinates, the focallength of the imaging lens 50, and the dimension of the imaging pixel60A1 to derive the direction of a plane specified by the plane equation.Meanwhile, the first characteristic pixel coordinates used in theprocessing of step 372 are the first characteristic pixel coordinatesacquired in the processing of step 354, and are equivalent to the threecharacteristic pixel coordinates described in the first embodiment. Inaddition, the second characteristic pixel coordinates used in theprocessing of step 372 are the second characteristic pixel coordinatesacquired in the processing of step 370, and are equivalent to thecorresponding characteristic pixel coordinates described in the firstembodiment.

As described above, in the distance measurement device 10C, the threecharacteristic pixels are designated through the touch panel 88 in thefirst captured image, and the first characteristic pixel coordinates forspecifying the designated three characteristic pixels are acquired bythe acquisition unit 172 (step 354). In addition, the threecharacteristic pixels corresponding to the three characteristic pixelsof the first captured image are designated through the touch panel 88 inthe second captured image (step 366: Y). In addition, the secondcharacteristic pixel coordinates for specifying the three characteristicpixels designated through the touch panel 88 in the second capturedimage are acquired by the acquisition unit 172 (step 370). An imagingposition distance is calculated by the derivation unit 174 on the basisof the attention pixel coordinates, the corresponding attention pixelcoordinates, the first characteristic pixel coordinates, the secondcharacteristic pixel coordinates, the focus position coordinates, thefocal length of the imaging lens 50, and the dimension of the imagingpixel 60A1. Therefore, according to the distance measurement device 10C,it is possible to calculate the imaging position distance on the basisof the first characteristic pixel coordinates and the secondcharacteristic pixel coordinates which are acquired in accordance withthe user's intention.

Fourth Embodiment

In the above-described embodiments, a description has been given of acase where distance measurement is performed at a first position out ofthe first position and a second position, but a description will begiven of a case where distance measurement is also performed at thesecond position in a fourth embodiment. Meanwhile, in the fourthembodiment, the same components as those described in theabove-described embodiments will be denoted by the same referencenumerals and signs, and a description thereof will be omitted.

A distance measurement device 10D according to the fourth embodiment isdifferent from the distance measurement device 10A in that an imagingposition distance calculation program 180 is stored in a secondarystorage unit 104 instead of an imaging position distance calculationprogram 106. In addition, the distance measurement device 10D isdifferent from the distance measurement device 10A in that athree-dimensional coordinate calculation program 181 is stored in thesecondary storage unit 104 instead of a three-dimensional coordinatecalculation program 108.

A CPU 100 executes the imaging position distance calculation program 180and a three-dimensional coordinate calculation program 181 to beoperated as an acquisition unit 182, a derivation unit 184, and acontrol unit 185 as illustrated in FIG. 9 as an example.

The acquisition unit 182 corresponds to the acquisition unit 154described in the second embodiment, the derivation unit 184 correspondsto the derivation unit 112 described in the first embodiment, and thecontrol unit 185 corresponds to the control unit 156 described in thesecond embodiment. Meanwhile, in the fourth embodiment, for convenienceof description, different portions from the acquisition unit 154described in the second embodiment will be described with regard to theacquisition unit 182. In the fourth embodiment, for convenience ofdescription, different portions from the derivation unit 112 describedin the first embodiment will be described with regard to the derivationunit 184. Further, in the fourth embodiment, for convenience ofdescription, different portions from the control unit 156 described inthe second embodiment will be described with regard to the control unit185.

The acquisition unit 182 further acquires a reference distance, ascompared to the acquisition unit 154. The “reference distance” asmentioned herein refers to a distance which is measured on the basis ofa laser beam emitted by a distance measurement unit 12 at a secondmeasurement position.

The derivation unit 184 derives a reference imaging position distancewhich is a distance between a first imaging position and a secondimaging position, on the basis of attention pixel coordinates, threecharacteristic pixel coordinates, reference emission positioncoordinates, a focal length of an imaging lens 50, and a dimension of animaging pixel 60A1. The derivation unit 184 adjusts the imaging positiondistance with reference to the derived reference imaging positiondistance to derive a final imaging position distance which is finallyadopted as the distance between the first imaging position and thesecond imaging position.

In addition, the derivation unit 184 derives designated pixelthree-dimensional coordinates on the basis of the derived final imagingposition distance. The designated pixel three-dimensional coordinatesaccording to the fourth embodiment are an example of final designatedpixel real space coordinates according to the technique of thisdisclosure. The final designated pixel real space coordinates refer tothree-dimensional coordinates which are finally adopted as thethree-dimensional coordinates which are coordinates on the real space ofan attention pixel (see step 206 illustrated in FIG. 21) which is anexample of a designated pixel according to the technique of thisdisclosure.

Next, an imaging position distance calculation process realized by theCPU 100 executing the imaging position distance calculation program 180will be described with reference to FIG. 26, as the operation ofportions of the distance measurement device 10D according to thetechnique of this disclosure. Meanwhile, the same steps as those in theflowchart illustrated in FIG. 12 will be denoted by the same stepnumbers, and a description thereof will be omitted.

The flowchart illustrated in FIG. 26 is different from the flowchartillustrated in FIG. 12 in that steps 400 and 402 are provided instead ofsteps 218 and 222. In addition, the flowchart illustrated in FIG. 26 isdifferent from the flowchart illustrated in FIG. 12 in that steps 404 to416 are provided instead of steps 232 to 236. In addition, the flowchartillustrated in FIG. 26 is also the continuation of the flowchartillustrated in FIG. 24.

In step 400 illustrated in FIG. 26, the acquisition unit 182 determineswhether or not measurement and imaging of a distance at the secondposition have been executed by the distance measurement device 10D. Instep 400, in a case where measurement and imaging of a distance at thesecond position have not been executed by the distance measurementdevice 10D, the determination result is negative, and the processproceeds to step 220. In step 400, in a case where measurement andimaging of a distance at the second position have been executed by thedistance measurement device 10D, the determination result is positive,and the process proceeds to step 402.

In step 402, the acquisition unit 182 acquires a reference distancewhich is a distance measured at the second position and a secondcaptured image signal indicating a second captured image which isobtained by performing imaging at the second position, and then theprocess proceeds to step 224.

In step 404, the derivation unit 184 decides a first plane equationwhich is the plane equation shown in Expression (3) on the basis of theemission position coordinates calculated in the processing of step 214,and then the process proceeds to step 406.

In step 406, the derivation unit 184 calculates an imaging positiondistance on the basis of the attention pixel coordinates, thecorresponding attention pixel coordinates, the focal length of theimaging lens 50, the dimension of the imaging pixel 60A1, the firstplane equation, and Expression (1), and then the process proceeds tostep 408.

In step 408, the derivation unit 184 calculates reference emissionposition coordinates on the basis of Expression (2) from the referencedistance acquired by the acquisition unit 182 in the processing of step402, a half angle of view α, an emission angle β, and a distance betweenreference points M, and then the process proceeds to step 410.Meanwhile, the reference distance used in the processing of step 408 isa distance corresponding to the distance L described in the firstembodiment.

In step 410, the derivation unit 184 decides a second plane equationwhich is the plane equation shown in Expression (3) on the basis of thereference emission position coordinates derived in the processing ofstep 408, and then the process proceeds to step 412. That is, in step410, the derivation unit 184 substitutes a, b, and c derived in theprocessing of step 230 and the reference emission position coordinatesderived in the processing of step 408 for Expression (3) to decide d inExpression (3). Since a, b, and c in Expression (3) are derived in theprocessing of step 230, the second plane equation is decided when d inExpression (3) is decided in the processing of step 410.

In step 412, the derivation unit 184 derives a reference imagingposition distance on the basis of the attention pixel coordinates, thecorresponding attention pixel coordinates, the focal length of theimaging lens 50, the dimension of the imaging pixel 60A1, the secondplane equation, and Expression (1), and then the process proceeds tostep 414. Meanwhile, the reference imaging position distance isequivalent to “B” shown in Expression (1), and is calculated bysubstituting (X, Y, Z) in Expression (1), for which the attention pixelcoordinates, the corresponding attention pixel coordinates, the focallength of the imaging lens 50, and the dimension of the imaging pixel60A1 are substituted, for the second plane equation.

In step 414, the derivation unit 184 adjusts the imaging positiondistance calculated in the processing of step 406 with reference to thereference imaging position distance calculated in the processing of step412 to calculate the final imaging position distance, and then theprocess proceeds to step 416. Here, the adjustment of the imagingposition distance refers to, for example, the obtainment of an averagevalue between the imaging position distance and the reference imagingposition distance, the multiplication of the average value between theimaging position distance and the reference imaging position distanceand a first adjustment coefficient, or the multiplication of the imagingposition distance and a second adjustment coefficient.

Meanwhile, both the first adjustment coefficient and the secondadjustment coefficient are, for example, coefficients which are uniquelydetermined in accordance with the reference imaging position distance.The first adjustment coefficient is derived from, for example, acorrespondence table in which the reference imaging position distanceand the first adjustment coefficient are associated with each other inadvance, or a computational expression in which the reference imagingposition distance is set to be an independent variable and the firstadjustment coefficient is set to be a dependent variable. The secondadjustment coefficient is similarly derived. The correspondence tableand the computational expression are derived from a derivation table ora computational expression which is derived from results of experimentperformed by the real machine of the distance measurement device 10D orcomputer simulation based on design specifications of the distancemeasurement device 10D at the stage before the shipment of the distancemeasurement device 10D.

Accordingly, examples of the final imaging position distance include anaverage value between the imaging position distance and the referenceimaging position distance, a value obtained by multiplying the averagevalue between the imaging position distance and the reference imagingposition distance by the first adjustment coefficient, and a valueobtained by multiplying the imaging position distance by the secondadjustment coefficient.

In step 416, the control unit 185 displays the final imaging positiondistance calculated in the processing of step 414 on the display unit 86so as to be superimposed on the second captured image, as illustrated inFIG. 27 as an example. In step 416, the control unit 185 stores thefinal imaging position distance calculated in the processing of step 414in a predetermined storage region, and then terminates the imagingposition distance calculation process.

Next, reference will be made to FIG. 28 to describe a three-dimensionalcoordinate calculation process realized by the CPU 100 executing thethree-dimensional coordinate calculation program 181 in a case where athree-dimensional coordinate calculation button 90G is turned on.

In the three-dimensional coordinate calculation process illustrated inFIG. 28, first, the derivation unit 184 determines whether or not thefinal imaging position distance has been already calculated in theprocessing of step 414 included in the imaging position distancecalculation process, in step 450. In step 450, in a case where the finalimaging position distance has not been calculated in the processing ofstep 414 included in the imaging position distance calculation process,the determination result is negative, and the process proceeds to step458. In step 450, in a case where the final imaging position distancehas been already calculated in the processing of step 414 included inthe imaging position distance calculation process, the determinationresult is positive, and the process proceeds to step 458.

In step 452, the derivation unit 184 determines whether or not acalculation start condition has been satisfied. In step 452, in a casewhere the calculation start condition has not been satisfied, thedetermination result is negative, and the process proceeds to step 458.In step 452, in a case where the calculation start condition has beensatisfied, the determination result is positive, and the processproceeds to step 454.

In step 454, the derivation unit 184 calculates the designated pixelthree-dimensional coordinates on the basis of the attention pixelcoordinates, the corresponding attention pixel coordinates, the finalimaging position distance, the focal length of the imaging lens 50, thedimension of the imaging pixel 60A1, and Expression (1), and then theprocess proceeds to step 456.

Meanwhile, in step 454, the designated pixel three-dimensionalcoordinates are calculated by substituting the attention pixelcoordinates, the corresponding attention pixel coordinates, the finalimaging position distance, the focal length of the imaging lens 50, andthe dimension of the imaging pixel 60A1 for Expression (1).

In step 456, the control unit 185 displays the designated pixelthree-dimensional coordinates calculated in the processing of step 454on the display unit 86 so as to be superimposed on the second capturedimage, as illustrated in FIG. 29 as an example. In step 456, the controlunit 185 stores the designated pixel three-dimensional coordinatescalculated in the processing of step 454 in a predetermined storageregion, and then terminates the three-dimensional coordinate calculationprocess.

Meanwhile, in the example illustrated in FIG. 29, (20160, 50132, 137810)corresponds to the designated pixel three-dimensional coordinatescalculated in the processing of step 454. In the example illustrated inFIG. 29, the designated pixel three-dimensional coordinates aredisplayed in proximity to the attention pixel 126.

In step 458, the derivation unit 112 determines whether or not acondition for terminating the three-dimensional coordinate calculationprocess has been satisfied. In step 458, in a case where the conditionfor terminating the three-dimensional coordinate calculation process hasnot been satisfied, the determination result is negative, and theprocess proceeds to step 450. In step 458, in a case where the conditionfor terminating the three-dimensional coordinate calculation process hasbeen satisfied, the determination result is positive, and thus thethree-dimensional coordinate calculation process is terminated.

As described above, in the distance measurement device 10D, a distancefrom the second position to the subject is measured, and a referencedistance which is the measured distance is acquired by the acquisitionunit 182 (step 402). In addition, the reference emission positioncoordinates are calculated by the derivation unit 184 on the basis ofthe reference distance (step 408). In addition, the reference imagingposition distance is calculated by the derivation unit 184 on the basisof the attention pixel coordinates, the corresponding attention pixelcoordinates, the three characteristic pixel coordinates, thecorresponding characteristic pixel coordinates, the reference emissionposition coordinates, the focal length of the imaging lens 50, and thedimension of the imaging pixel 60A1 (step 406). The imaging positiondistance is adjusted by the derivation unit 184 with reference to thereference imaging position distance, and thus the final imaging positiondistance is calculated (step 414). Therefore, according to the distancemeasurement device 10D, it is possible to calculate a distance betweenthe first imaging position and the second imaging position with a highlevel of accuracy, as compared to a case where the reference imagingposition distance is not used.

In the distance measurement device 10D, the designated pixelthree-dimensional coordinates are calculated on the basis of the finalimaging position distance calculated in the imaging position distancecalculation process (see FIG. 28). Therefore, according to the distancemeasurement device 10D, it is possible to calculate the designated pixelthree-dimensional coordinates with a high level of accuracy, as comparedto a case where the final imaging position distance is not used.

Further, in the distance measurement device 10D, the designated pixelthree-dimensional coordinates are specified on the basis of theattention pixel coordinates, the corresponding attention pixelcoordinates, the final imaging position distance, the focal length ofthe imaging lens 50, and the dimension of the imaging pixel 60A1 (seeExpression (1)). Therefore, according to the distance measurement device10D, it is possible to derive the designated pixel three-dimensionalcoordinates with a high level of accuracy, as compared to a case wherethe designated pixel three-dimensional coordinates are not specified onthe basis of the final imaging position distance, the attention pixelcoordinates, the corresponding attention pixel coordinates, the focallength of the imaging lens 50, and the dimension of the imaging pixel60A1.

Meanwhile, in the fourth embodiment, a distance measured on the basis ofa laser beam emitted from the second position is set to be the referencedistance, but the technique of this disclosure is not limited thereto.For example, a distance measured on the basis of a laser beam emittedfrom the first position may be set to be the reference distance.

Fifth Embodiment

In the above-described embodiments, a description has been given of acase where an imaging position distance and the like are derived by onedistance measurement device, but a description will be given of a casewhere an imaging position distance and the like are derived by two of adistance measurement devices and a personal computer (hereinafter,referred to as a PC) in a fifth embodiment. Meanwhile, PC stands for aPersonal Computer. Meanwhile, in the fifth embodiment, the samecomponents as those described in the above-described embodiments will bedenoted by the same reference numerals and signs, and a descriptionthereof will be omitted.

As illustrated in FIG. 30 as an example, an information processingsystem 500 according to the fifth embodiment includes distancemeasurement devices 10E1 and 10E2, and a PC 502. Meanwhile, in the fifthembodiment, the PC 502 can communicate with the distance measurementdevices 10E1 and 10E2. In the fifth embodiment, the PC 502 is an exampleof an information processing device according to the technique of thisdisclosure.

As illustrated in FIG. 30 as an example, the distance measurement device10E1 is disposed at a first position, and the distance measurementdevice 10E2 is disposed at a second position different from the firstposition.

As illustrated in FIG. 31 as an example, the distance measurementdevices 10E1 and 10E2 have the same configuration. Meanwhile,hereinafter, the distance measurement devices 10E1 and 10E2 will bereferred to as a “distance measurement device 10E” in a case where it isnot necessary to give a description by distinguishing between thedistance measurement devices.

The distance measurement device 10E is different from the distancemeasurement device 10A in that an imaging device 15 is provided insteadof the imaging device 14. The imaging device 15 is different from theimaging device 14 in that an imaging device main body 19 is providedinstead of the imaging device main body 18.

The imaging device main body 19 is different from the imaging devicemain body 18 in that a communication I/F 83 is provided. Thecommunication I/F 83 is connected to a bus line 84, and is operatedunder the control of a main control unit 62.

The communication I/F 83 is connected to a communication network (notshown) such as the Internet, and transmits and receives variousinformation to and from the PC 502 connected to the communicationnetwork.

As illustrated in FIG. 32 as an example, the PC 502 includes a maincontrol unit 503. The main control unit 503 includes a CPU 504, aprimary storage unit 506, and a secondary storage unit 508. The CPU 504,the primary storage unit 506, and the secondary storage unit 508 areconnected to each other through a bus line 510.

In addition, the PC 502 includes a communication I/F 512. Thecommunication I/F 512 is connected to the bus line 510, and is operatedunder the control of the main control unit 503. The communication I/F512 is connected to the communication network, and transmits andreceives various information to and from the distance measurement device10E connected to the communication network.

In addition, the PC 502 includes a reception unit 513 and a display unit514. The reception unit 513 is connected to the bus line 510 through areception I/F (not shown), and the reception I/F outputs an instructioncontent signal indicating contents of an instruction received by thereception unit 513 to the main control unit 503. Meanwhile, thereception unit 513 is realized by, for example, a keyboard, a mouse, anda touch panel.

The display unit 514 is connected to the bus line 510 through a displaycontrol unit (not shown), and displays various information under thecontrol of the display control unit. Meanwhile, the display unit 514 isrealized by, for example, an LCD.

The secondary storage unit 508 stores the imaging position distancecalculation program 106 (150, 168, 180) and the three-dimensionalcoordinate calculation program 108 (181) which are described in theabove-described embodiments. Meanwhile, hereinafter, for convenience ofdescription, the imaging position distance calculation programs 106,150, 168, and 180 will be referred to as an “imaging position distancecalculation program” without a reference numeral in a case where it isnot necessary to give a description by distinguishing between theimaging position distance calculation programs. In addition,hereinafter, for convenience of description, the three-dimensionalcoordinate calculation programs 108 and 181 will be referred to as a“three-dimensional coordinate calculation program” without a referencenumeral in a case where it is not necessary to give a description bydistinguishing between the three-dimensional coordinate calculationprograms.

The CPU 504 acquires a first captured image signal, attention pixelcoordinates, a distance, and the like from the distance measurementdevice 10E1 through the communication I/F 512. In addition, the CPU 504acquires a second captured image signal and the like from the distancemeasurement device 10E2 through the communication I/F 512.

The CPU 504 reads out the imaging position distance calculation programand the three-dimensional coordinate calculation program from thesecondary storage unit 508 and develops the read-out imaging positiondistance calculation program and three-dimensional coordinatecalculation program to the primary storage unit 506 to execute theimaging position distance calculation program and the three-dimensionalcoordinate calculation program. Meanwhile, hereinafter, for convenienceof description, the imaging position distance calculation program andthe three-dimensional coordinate calculation program are collectivelyreferred to as a “calculation program”.

The CPU 100 executes the calculation programs to be operated as theacquisition unit 110 (154, 172, 182), the derivation unit 112 (174,184), and the control unit 114 (156, 176, 185).

Accordingly, in the information processing system 500, the PC 502acquires the first captured image signal, second captured image signal,the attention pixel coordinates, the distance, and the like from thedistance measurement device 10E through the communication I/F 512 andthen executes the calculation programs, and thus the same operations andeffects as those in the above-described embodiments are obtained.

Sixth Embodiment

In the first embodiment, a description has been given of a case wherethe distance measurement device 10A is realized by the distancemeasurement unit 12 and the imaging device 14, but a description will begiven of a distance measurement device 10F which is realized by furtherincluding a smart device 602 in a sixth embodiment. Meanwhile, in thesixth embodiment, the same components as those in the above-describedembodiments will be denoted by the same reference numerals and signs,and a description thereof will be omitted, and only different portionsfrom the above-described embodiments will be described.

As illustrated in FIG. 33 as an example, the distance measurement device10F according to the third embodiment is different from the distancemeasurement device 10A according to the first embodiment in that animaging device 600 is provided instead of the imaging device 14. Inaddition, the distance measurement device 10F is different from thedistance measurement device 10A in that a smart device 602 is provided.

The imaging device 600 is different from the imaging device 14 in thatan imaging device main body 603 is provided instead of the imagingdevice main body 18.

The imaging device main body 603 is different from the imaging devicemain body 18 in that a wireless communication unit 604 and a wirelesscommunication antenna 606 are provided.

The wireless communication unit 604 is connected to a bus line 84 andthe wireless communication antenna 606. The main control unit 62 outputstransmission target information, which is information to be transmittedto the smart device 602, to the wireless communication unit 604.

The wireless communication unit 604 transmits the transmission targetinformation, which is input from the main control unit 62, to the smartdevice 602 by radio waves through the wireless communication antenna606. In addition, when the radio waves from the smart device 602 arereceived by the wireless communication antenna 606, the wirelesscommunication unit 604 acquires a signal based on the received radiowaves, and outputs the acquired signal to the main control unit 62.

The smart device 602 includes a CPU 608, a primary storage unit 610, anda secondary storage unit 612. The CPU 608, the primary storage unit 610,and the secondary storage unit 612 are connected to a bus line 614.

The CPU 608 controls the entire distance measurement device 10F,inclusive of the smart device 602. The primary storage unit 610 is avolatile memory which is used as a work area and the like during theexecution of various programs. An example of the primary storage unit610 is a RAM. The secondary storage unit 612 is a non-volatile memorythat stores a control program for controlling the overall operation ofthe distance measurement device 10F, various parameters, and the like,inclusive of the smart device 602. An example of the secondary storageunit 612 is a flash memory or an EEPROM.

The smart device 142 includes a display unit 615, a touch panel 616, awireless communication unit 618, and a wireless communication antenna620.

The display unit 615 is connected to the bus line 614 through a displaycontrol unit (not shown), and displays various information under thecontrol of the display control unit. Meanwhile, the display unit 615 isrealized by, for example, an LCD.

The touch panel 616 is superimposed on a display screen of the displayunit 615, and receives a touch by an indicator. The touch panel 616 isconnected to the bus line 614 through a touch panel I/F (not shown), andoutputs positional information indicating a position touched by theindicator to the touch panel I/F. The touch panel I/F is operated inaccordance with an instruction of the CPU 608, and outputs thepositional information, which is input from the touch panel 616, to theCPU 608.

Soft keys equivalent to a measurement and imaging button 90A, an imagingbutton 90B, an imaging system operation mode switching button 90C, awide angle instruction button 90D, a telephoto instruction button 90E,an imaging position distance calculation button 90F, a three-dimensionalcoordinate calculation button 90G, and the like are displayed on thedisplay unit 615.

For example, as illustrated in FIG. 34, a measurement and imaging button90A1 functioning as the measurement and imaging button 90A is displayedon the display unit 615 as a soft key, and is pressed down by the userthrough the touch panel 616. In addition, for example, an imaging button90B1 functioning as the imaging button 90B is displayed on the displayunit 615 as a soft key, and is pressed down by the user through thetouch panel 616. In addition, for example, an imaging system operationmode switching button 90C1 functioning as the imaging system operationmode switching button 90C is displayed on the display unit 615 as a softkey, and is pressed down by the user through the touch panel 616.

In addition, for example, a wide angle instruction button 90D1functioning as the wide angle instruction button 90D is displayed on thedisplay unit 615 as a soft key, and is pressed down by the user throughthe touch panel 616. Further, for example, a telephoto instructionbutton 90E1 functioning as the telephoto instruction button 90E isdisplayed on the display unit 615 as a soft key, and is pressed down bythe user through the touch panel 616.

In addition, for example, an imaging position distance calculationbutton 90F1 functioning as the imaging position distance calculationbutton 90F is displayed on the display unit 615 as a soft key, and ispressed down by the user through the touch panel 616. In addition, forexample, a three-dimensional coordinate calculation button 90G1functioning as the three-dimensional coordinate calculation button 90Gis displayed on the display unit 615 as a soft key, and is pressed downby the user through the touch panel 616.

The wireless communication unit 618 is connected to the bus line 614 andthe wireless communication antenna 620. The wireless communication unit618 transmits a signal, which is input from the CPU 608, to the imagingdevice main body 603 by radio waves through the wireless communicationantenna 620. In addition, when the radio waves are received by thewireless communication antenna 620 from the imaging device main body603, the wireless communication unit 618 acquires a signal based on thereceived radio waves and outputs the acquired signal to the CPU 608.Therefore, the imaging device main body 603 is controlled by the smartdevice 602 through wireless communication performed between the smartdevice 602 and the imaging device main body 603.

The secondary storage unit 612 stores a calculation program. The CPU 608reads out the calculation program from the secondary storage unit 612and develops the read-out calculation program to the primary storageunit 610 to execute the calculation program.

The CPU 608 executes the calculation program to be operated as theacquisition unit 110 (154, 172, 182), the derivation unit 112 (174,184), and the control unit 114 (156, 176, 185). For example, the CPU 608executes the imaging position distance calculation program 106, and thusthe imaging position distance calculation process described in the firstembodiment is realized. In addition, for example, the CPU 608 executesthe three-dimensional coordinate calculation program 108, and thus thethree-dimensional calculation process described in the first embodimentis realized.

Therefore, in the distance measurement device 10F, the smart device 602executes the calculation program, and thus the same operations andeffects as those in the above-described embodiments are obtained. Inaddition, according to the distance measurement device 10F, it ispossible to reduce a load applied to the imaging device 600 in obtainingthe effects described in the above-described embodiments, as compared toa case where the imaging position distance calculation process and thethree-dimensional calculation process are executed by the imaging device600.

Meanwhile, in the above-described embodiments, a corresponding attentionpixel is specified by executing image analysis with a second capturedimage as an object to be analyzed, and corresponding attention pixelcoordinates for specifying the specified corresponding attention pixelare acquired (see step 226 illustrated in FIG. 12), but the technique ofthis disclosure is not limited thereto. For example, the user maydesignate a pixel corresponding to an attention pixel as thecorresponding attention pixel from the second captured image through thetouch panel 88.

In the above-described embodiments, a description has been given of acase where the derivation unit 112 (174, 184) calculates emissionposition coordinates, the direction of a plane, an imaging positiondistance, designated pixel three-dimensional coordinates, and the likeby using a computational expression, but the technique of thisdisclosure is not limited thereto. For example, the derivation unit 112(174, 184) may calculate emission position coordinates, the direction ofa plane, an imaging position distance, designated pixelthree-dimensional coordinates, and the like by using a table in which anindependent variable of the computational expression is set to be aninput and a dependent variable of the computational expression is set tobe an output.

In the above-described embodiments, a description has been given of acase where the calculation program is read out from the secondarystorage unit 104 (508,612), but the calculation program is notnecessarily stored in the secondary storage unit 104 (508,612) from thebeginning. For example, as illustrated in FIG. 35, the calculationprogram may be first stored in any portable storage medium 700 such as aSolid State Drive (SSD) or a Universal Serial Bus (USB) memory. In thiscase, the calculation program of the storage medium 700 is installed inthe distance measurement device 10A (10B, 10C, 10D, 10F) (hereinafter,referred to as “distance measurement device 10A and the like”) or the PC502, and the installed calculation program is executed by the CPU 100(608).

In addition, the calculation program may be stored in a storage unit ofanother computer or a server device connected to the distancemeasurement device 10A and the like or the PC 502 through acommunication network (not shown), and the calculation program may bedownloaded in accordance with requests of the distance measurementdevice 10A and the like. In this case, the downloaded calculationprogram is executed by the CPU 100 (608).

In the above-described embodiments, a description has been given of acase where various information such as an emission position mark 136, animaging position distance, and designated pixel three-dimensionalcoordinates is displayed on the display unit 86, but the technique ofthis disclosure is not limited thereto. For example, various informationmay be displayed on a display unit of an external device which is usedby being connected to the distance measurement device 10A and the likeor the PC 502. An example of the external device is a PC or aspectacles-type or wristwatch type wearable terminal device.

In the above-described embodiments, a description has been given of acase where the emission position mark 136, the imaging positiondistance, the designated pixel three-dimensional coordinates, and thelike are visibly displayed by the display unit 86, but the technique ofthis disclosure is not limited thereto. For example, audible displaysuch as the output of a sound using a sound reproducing device orpermanent visible display such as the output of printed matter using aprinter may be performed instead of the visible display or may beperformed in combination.

In the above-described embodiments, a description has been given of acase where the emission position mark 136, the imaging positiondistance, the designated pixel three-dimensional coordinates, and thelike are displayed on the display unit 86, but the technique of thisdisclosure is not limited thereto. For example, at least one of theemission position mark 136, the imaging position distance, thedesignated pixel three-dimensional coordinates, and the like may bedisplayed on a display unit (not shown) different from the display unit86, and the remainders may be displayed on the display unit 86. Theemission position mark 136, the imaging position distance, thedesignated pixel three-dimensional coordinates, and the like may beindividually displayed on a plurality of display units including thedisplay unit 86.

In the above-described embodiments, a laser beam has been described aslight for distance measurement. However, the technique of thisdisclosure is not limited thereto, and a directional light which isdirectional light may be used. For example, the directional light may bedirectional light beam obtained by a Light Emitting Diode ((LED) or aSuper Luminescent Diode ((SLD). It is preferable that directivity of thedirectional light beam is the same degree of directivity as that of thedirectivity of the laser beam and is usable in distance measurement, forexample, within a range between several meters and several kilometers.

In addition, the imaging position distance calculation process and thethree-dimensional coordinate calculation process described in theabove-described embodiments are just examples. Therefore, it is needlessto say that the deletion of unnecessary steps, the addition of newsteps, and the change of processing order may be performed withoutdeparting from the scope of the invention. In addition, each processingincluded in the imaging position distance calculation process and thethree-dimensional coordinate calculation process may be realized only bya hardware configuration such as ASIC, or may be realized by acombination of a software configuration and a hardware configurationusing a computer

In the above-described embodiments, for convenience of description, adescription has been given of a case where the distance measurement unit12 is mounted on the side surface of the imaging device main body 18included in the distance measurement device 10A and the like, but thetechnique of this disclosure is not limited thereto. For example, thedistance measurement unit 12 may be mounted on the upper surface or thelower surface of the imaging device main body 18. In addition, forexample, as illustrated in FIG. 36, a distance measurement device 10Gmay be applied instead of the distance measurement device 10A and thelike. As illustrated in FIG. 36 as an example, the distance measurementdevice 10G is different from the distance measurement device 10A and thelike in that a distance measurement unit 12A is provided instead of thedistance measurement unit 12 and an imaging device main body 18A isprovided instead of the imaging device main body 18.

In the example illustrated in FIG. 36, the distance measurement unit 12Ais accommodated in a housing 18A1 of the imaging device main body 18A,and objective lenses 32 and 38 are exposed from the housing 18A1 on thefront side (a side where the imaging lens 50 is exposed) of the distancemeasurement device 10G. In addition, it is preferable that the distancemeasurement unit 12A is disposed such that optical axes L1 and L2 areset to be at the same height. Meanwhile, an opening (not shown) throughwhich the distance measurement unit 12A can be inserted into and removedfrom the housing 18A1 may be formed in the housing 18A1.

Meanwhile, the half angle of view α used in the processing of step 214included in the imaging position distance calculation process accordingto the first embodiment and the half angle of view α used in theprocessing of step 408 included in the imaging position distancecalculation process according to the fourth embodiment are derived onthe basis of the following Expression (7). In Expression (7), “f₀”denotes a focal length.

$\begin{matrix}{\alpha = {{atan}\left\{ \frac{\left( {{dimension}\mspace{14mu}{of}\mspace{14mu}{imaging}\mspace{14mu}{pixel}} \right)}{2 \times f_{0}} \right\}}} & (7)\end{matrix}$

All the documents, patent applications, and technical specificationsdescribed in the present specification are incorporated into the presentspecification by reference, to the same extent as in a case where theindividual documents, patent applications, and technical specificationswere specifically and individually described as being incorporated byreference.

With regard to the above-described embodiments, the following appendixeswill be further disclosed.

(Appendix 1)

-   -   An information processing device including:    -   an acquisition unit that acquires a first captured image        obtained by imaging a subject from a first imaging position, a        second captured image obtained by imaging the subject from a        second imaging position different from the first imaging        position, and a distance from one of a position corresponding to        the first imaging position and a position corresponding to the        second imaging position to the subject, the distance being        measured by emitting directional light, which has directivity,        to the subject and receiving a reflected light of the        directional light beam; and    -   a derivation unit that derives an imaging position distance        which is a distance between the first imaging position and the        second imaging position, on the basis of designated pixel        coordinates which are coordinates for specifying designated        pixels designated as pixels corresponding to a position on a        real space in each of the first captured image and the second        captured image which are acquired by the acquisition unit, a        plurality of pixel coordinates being a plurality of coordinates        for specifying a plurality of pixels of more than three pixels        which are present in the same planar region as an emission        position irradiated with the directional light beam on the real        space and correspond to the position on the real space in each        of the first captured image and the second captured image which        are acquired by the acquisition unit, emission position        coordinates which specifies the emission position on the real        space and are derived on the basis of the distance acquired by        the acquisition unit, a focal length of an imaging lens used for        the imaging of the subject, and dimensions of imaging pixels        included in an imaging pixel group for imaging the subject.

(Appendix 2)

-   -   The information processing device according to Appendix 1,    -   wherein the derivation unit derives designated pixel real space        coordinates which are coordinates of the designated pixels on        the real space on the basis of the derived imaging position        distance.

(Appendix 3)

-   -   The information processing device according to Appendix 2,    -   wherein the designated pixel real space coordinates are        specified on the basis of the imaging position distance, the        designated pixel coordinates, the focal length, and the        dimensions.

(Appendix 4)

-   -   The information processing device according to any one of        Appendixes 1 to 3,    -   wherein the derivation unit derives a direction of a plane,        including coordinates on the real space which correspond to the        plurality of pixel coordinates, which is specified by a plane        equation indicating the plane on the basis of the plurality of        pixel coordinates, the focal length, and the dimensions, decides        the plane equation on the basis of the derived direction and the        emission position coordinates, and derives the imaging position        distance on the basis of the decided plane equation, the        designated pixel coordinates, the focal length, and the        dimensions.

(Appendix 5)

-   -   The information processing device according to any one of        Appendixes 1 to 4,    -   wherein the plurality of pixels are designated by first pixel        designation information for designating a pixel from each of the        first captured image and the second captured image, the first        pixel designation information being received by a first        reception unit receiving the first pixel designation        information, and    -   wherein the acquisition unit acquires a plurality of coordinates        for specifying the plurality of pixels designated in accordance        with the first pixel designation information as the plurality of        pixel coordinates, and the derivation unit derives the imaging        position distance on the basis of the designated pixel        coordinates, the plurality of pixel coordinates acquired by the        acquisition unit, the emission position coordinates, the focal        length, and the dimensions.

(Appendix 6)

-   -   The information processing device according to any one of        Appendixes 1 to 4,    -   wherein the acquisition unit acquires a plurality of        coordinates, as the plurality of pixel coordinates, for        specifying a plurality of characteristic pixels more than three        pixels which are present in the same planar region as the        emission position on the real space and correspond to the        position on the real space in each of the first captured image        and the second captured image, and    -   wherein the derivation unit derives the imaging position        distance on the basis of the designated pixel coordinates, the        plurality of pixel coordinates acquired by the acquisition unit,        the emission position coordinates, the focal length, and the        dimensions.

(Appendix 7)

-   -   The information processing device according to Appendix 6,    -   wherein the plurality of characteristic pixels are a        predetermined number of pixels more than three pixels which are        present in the same planar region as the emission position on        the real space and correspond to the position on the real space        in each of the first captured image and the second captured        image, and are a plurality of pixels for maximizing an area        surrounded.

(Appendix 8)

-   -   The information processing device according to Appendix 6,        further including:    -   a first control unit that performs control of displaying at        least one of the first captured image and the second captured        image on a first display unit, and displaying a corresponding        region corresponding to the same planar region as the emission        position within a display region so as to be distinguishable        from the other regions,    -   wherein the acquisition unit acquires a plurality of coordinates        for specifying the plurality of characteristic pixels as the        plurality of characteristic pixels coordinates, from a portion        of the corresponding region designated in accordance with region        designation information received by a second reception unit        receiving the region designation information for designating a        portion of the corresponding region in a state where the        corresponding region is displayed on the first display unit.

(Appendix 9)

-   -   The information processing device according to any one of        Appendixes 1 to 8,    -   wherein the designated pixel related to one of the first        captured image and the second captured image is a pixel which is        designated in accordance with second pixel designation        information received by a third reception unit receiving the        second pixel designation information for designating a pixel        from one of the first captured image and the second captured        image, and    -   wherein the designated pixel related to the other one of the        first captured image and the second captured image is a pixel        which is included in the other one of the first captured image        and the second captured image and corresponds to a position of        the pixel designated in accordance with the second pixel        designation information on the real space.

(Appendix 10)

-   -   The information processing device according to any one of        Appendixes 1 to 9, further including:    -   a measurement unit that measures the distance by emitting the        directional light beam and receiving the reflected light,    -   wherein the acquisition unit acquires the distance measured by        the measurement unit.

(Appendix 11)

-   -   The information processing device according to any one of        Appendixes 1 to 10, further including:    -   an imaging unit that images the subject,    -   wherein the acquisition unit that acquires the first captured        image obtained by imaging the subject by the imaging unit from        the first imaging position, and the second captured image        obtained by imaging the subject by the imaging unit from the        second imaging position.

(Appendix 12)

-   -   The information processing device according to any one of        Appendixes 1 to 11,    -   wherein the acquisition unit further acquires a reference        distance to the subject which is measured by emitting the        directional light beam to the subject from the other one of the        position corresponding to the first imaging position and the        position corresponding to the second imaging position and        receiving the reflected light of the directional light beam,    -   wherein the derivation unit further derives a reference imaging        position distance which is the distance between the first        imaging position and the second imaging position on the basis of        the designated pixel coordinates, the plurality of pixel        coordinates, reference emission position coordinates for        specifying the emission position on the real space and derived        on the basis of the reference distance acquired by the        acquisition unit, the focal length, and the dimensions, and        adjusts the imaging position distance with reference to the        derived reference imaging position distance to derive a final        imaging position distance which is finally adopted as the        distance between the first imaging position and the second        imaging position.

(Appendix 13)

-   -   The information processing device according to Appendix 12,    -   wherein the derivation unit derives final designated pixel real        space coordinates which are finally adopted as the coordinates        of the designated pixels on the real space, on the basis of the        derived final imaging position distance.

(Appendix 14)

-   -   The information processing device according to Appendix 13,    -   wherein the final designated pixel real space coordinates are        specified on the basis of the final imaging position distance,        the designated pixel coordinates, the focal length, and the        dimensions.

(Appendix 15)

-   -   The information processing device according to any one of        Appendixes 1 to 14, further including:    -   a second control unit that performs control of displaying        derivation results of the derivation unit on a second display        unit.

(Appendix 16)

-   -   An information processing method including:    -   acquiring a first captured image obtained by imaging a subject        from a first imaging position, a second captured image obtained        by imaging the subject from a second imaging position different        from the first imaging position, and a distance from one of a        position corresponding to the first imaging position and a        position corresponding to the second imaging position to the        subject, the distance being measured by emitting directional        light, which has directivity, to the subject and receiving a        reflected light of the directional light beam; and    -   deriving an imaging position distance which is a distance        between the first imaging position and the second imaging        position, on the basis of designated pixel coordinates which are        coordinates for specifying designated pixels designated as        pixels corresponding to a position on a real space in each of        the acquired first captured image and second captured image, a        plurality of pixel coordinates being a plurality of coordinates        for specifying a plurality of pixels of more than three pixels        which are present in the same planar region as an emission        position irradiated with the directional light beam on the real        space and correspond to the position on the real space in each        of the acquired first captured image and second captured image,        emission position coordinates which specifies the emission        position on the real space and are derived on the basis of the        acquired distance, a focal length of an imaging lens used for        the imaging of the subject, and dimensions of imaging pixels        included in an imaging pixel group for imaging the subject.

(Appendix 17)

-   -   A program causing a computer to execute processes of:    -   acquiring a first captured image obtained by imaging a subject        from a first imaging position, a second captured image obtained        by imaging the subject from a second imaging position different        from the first imaging position, and a distance from one of a        position corresponding to the first imaging position and a        position corresponding to the second imaging position to the        subject, the distance being measured by emitting directional        light, which has directivity, to the subject and receiving a        reflected light of the directional light beam; and    -   deriving an imaging position distance which is a distance        between the first imaging position and the second imaging        position, on the basis of designated pixel coordinates which are        coordinates for specifying designated pixels designated as        pixels corresponding to a position on a real space in each of        the acquired first captured image and second captured image, a        plurality of pixel coordinates being a plurality of coordinates        for specifying a plurality of pixels of more than three pixels        which are present in the same planar region as an emission        position irradiated with the directional light beam on the real        space and correspond to the position on the real space in each        of the acquired first captured image and second captured image,        emission position coordinates which specifies the emission        position on the real space and are derived on the basis of the        acquired distance, a focal length of an imaging lens used for        the imaging of the subject, and dimensions of imaging pixels        included in an imaging pixel group for imaging the subject.

What is claimed is:
 1. An information processing device comprising: adisplay; a processor; and a memory that is connected to or built in theprocessor, wherein the processor is configured to acquire a firstcaptured image obtained by imaging a subject from a first imagingposition, a second captured image obtained by imaging the subject from asecond imaging position different from the first imaging position, afirst distance from a position corresponding to the first imagingposition to the subject, and a second distance from a positioncorresponding to the second imaging position to the subject, display thefirst captured image or the second captured image on the display,specify a first position designated on the display on which the firstcaptured image is displayed and a second position designated on thedisplay on which the second captured image is displayed, and derive adistance in a real space based on the first distance, the seconddistance, the first position and the second position.
 2. The informationprocessing device according to claim 1, wherein the processor furtherconfigured to specify a position in the second captured imagecorresponding to the first position in the real space.
 3. Theinformation processing device according to claim 2, wherein the displayhas a function that detects a touch of an user, and the processordisplays a plurality of soft keys including a key used for a distancemeasurement and a key used for imaging.
 4. The information processingdevice according to claim 3, wherein the plurality of soft keys includemore than three soft keys, and the processor displays the plurality ofsoft keys on a same straight line.
 5. The information processing deviceaccording to claim 4, wherein the same straight line is parallel to oneside of the display.
 6. The information processing device according toclaim 2, wherein the processor displays the distance in the real spaceon the display so as to be superimposed on the second captured image. 7.The information processing device according to claim 1, wherein theprocessor displays the first position in a different manner from otherpixels on the display.
 8. The information processing device according toclaim 7, wherein the display has a function that detects a touch of anuser, and the processor displays a plurality of soft keys including akey used for a distance measurement and a key used for imaging.
 9. Theinformation processing device according to claim 8, wherein theplurality of soft keys include more than three soft keys, and theprocessor displays the plurality of soft keys on a same straight line.10. The information processing device according to claim 7, wherein theprocessor displays the distance in the real space on the display so asto be superimposed on the second captured image.
 11. The informationprocessing device according to claim 1, wherein the processor displaysthe second position in a different manner from other pixels on thedisplay.
 12. The information processing device according to claim 1,wherein the display has a function that detects a touch of an user, andthe processor displays a plurality of soft keys including a key used fora distance measurement and a key used for imaging.
 13. The informationprocessing device according to claim 12, wherein the plurality of softkeys include more than three soft keys, and the processor displays theplurality of soft keys on a same straight line.
 14. The informationprocessing device according to claim 13, wherein the same straight lineis parallel to one side of the display.
 15. The information processingdevice according to claim 1, wherein the processor displays the distancein the real space on the display so as to be superimposed on the secondcaptured image.
 16. The information processing device according to claim1, wherein the processor acquires the first distance on the basis of atiming when directional light which has directivity is emitted to thesubject from the position corresponding to the first imaging positionand a timing when reflected light of the directional light from theposition corresponding to the first imaging position is received, andacquires the second distance on the basis of a timing when directionallight which has directivity is emitted to the subject from the positioncorresponding to the second imaging position and a timing when reflectedlight of the directional light from the position corresponding to thesecond imaging position is received.
 17. The information processingdevice according to claim 16, further comprising: a body that houses theprocessor; an image sensor that images the subject; a first lens that isused for emitting the directional light; a second lens that is used forreceiving reflected light of the directional light; and a third lensthat is used for imaging the subject, the first lens, the second lens,and the third lens are provided on a same surface of the body and a samestraight line.
 18. The information processing device according to claim1, further comprising: an image sensor that images the subject, whereinthe processor acquires the first captured image obtained by imaging thesubject by the image sensor from the first imaging position, and thesecond captured image obtained by imaging the subject by the imagesensor from the second imaging position.
 19. An information processingmethod comprising: acquiring a first captured image obtained by imaginga subject from a first imaging position, a second captured imageobtained by imaging the subject from a second imaging position differentfrom the first imaging position, a first distance from a positioncorresponding to the first imaging position to the subject, and a seconddistance from a position corresponding to the second imaging position tothe subject; displaying the first captured image or the second capturedimage on a display; specifying a first position designated on thedisplay on which the first captured image is displayed and a secondposition designated on the display on which the second captured image isdisplayed; and deriving a distance in a real space based on the firstdistance, the second distance, the first position and the secondposition.
 20. A non-transitory computer-readable storage medium storinga program for causing a computer to execute a process comprising:acquiring a first captured image obtained by imaging a subject from afirst imaging position, a second captured image obtained by imaging thesubject from a second imaging position different from the first imagingposition, a first distance from a position corresponding to the firstimaging position to the subject, and a second distance from a positioncorresponding to the second imaging position to the subject; displayingthe first captured image or the second captured image on a display;specifying a first position designated on the display on which the firstcaptured image is displayed and a second position designated on thedisplay on which the second captured image is displayed; and deriving adistance in a real space based on the first distance, the seconddistance, the first position and the second position.